A fair point. This is why I call the category “trust-minimized” rather than “trustless.” And why, in the sentence you quote, I focus on trusting specific transaction verifiers vs. broadly trusting the Bitcoin codebase.
One of the central claims of my book is that we are dealing with a new architecture of trust, not the elimination of trust. A second major claim, consistent with what you say here, is that blockchain and cryptocurrencies need to engage with legal, regulatory, and governance mechanisms. As you point out, there is a need for recourse when thing go wrong.