Here, had been trying in Past Perfect Continuous Tense is correct because it
is used to indicate an action in the past that began before a certain point in
the past and continued up untill that time. Here, ten minutes in the sentence indicate an action in the past that began before a certain point in the past.
'Were trying' in past continuous is incorrect because it
is used to indicate uncompleted action in the past (with or without time
reference).
'Had tried' in past perfect is incorrect because it
is used to indicate a completed action of the past that happened before another
event took place.
'Tried' in past simple is incorrect because it
indicates an action that has already been completed.