The fact it was only a year ago takes Ted aback. He knew about Andrew, but...

When Sherlock says that she was also kidnapped in connection with it, Ted instinctively reaches out to take Joan's hand. Both to comfort, but also just... to hang onto her? To protect her? As if he can protect her from something that happened in the past.