"Just wanted to check how you are. I know it must be shit but...there's levels of shit, right." he tries to joke.

He hands Sherlock the tea.

"Uhm...I should have asked, do...Do you want my key back?" He says, nervous; "I mean, it's up to you if you want me to be able to come and go here and it's more than fair if you'd rather I didn't..." Ella did tell him as much. Let Sherlock set the boundaries.