The correct option is B while
A conjunction combines or connects two sentences as one. When one of them depends on the other for its full meaning, that clause is the subordinate clause and the clause it depends on is the main clause. Here, we require a time-related conjunction as both clauses are in terms of the events (he was talking, he was holding his new born baby) occurring at a particular time. From the options, 'when' demonstrates an interruption, a very short lapse in time between the two activities, which is not the case here. 'While' is used when two activities are taking place simultaneously, and here that is exactly what is happening. Hence option B is correct and A is incorrect.
'But' is a coordinating conjunction used when both clauses are independent and do not depend on the other for their full meaning, and is used to express a contrast in meaning between these clauses. But here we don't need a coordinating conjunction, and the clauses are not contrast-related, hence option C is incorrect. 'For' is also a coordinating conjunction, used when one clause is inferred or deduced from another, and that isn't the case here, hence option D is incorrect.