Why are so many permits required for even simple changes or maintenance to your home?

I live in Washington, and they're requiring me to get a permit to replace a window? It's a simple swap with no structural or size opening changes, and the permit costs $50.

It feels like a way for the city or county to take our money. They give themselves the authority to require permits, inspections, and add on permit fees (usually a percentage or a minimum fee—$175 per permit in my county). They require a permit to add, move, or remove a shed. This system also keeps their tax assessment database up to date with any changes on your property (new shed, new kitchen, two bathrooms, new roof, new windows, etc.). Oh, that new improvement looks nice! But now, your property taxes go up by $500 per year due to those "improvements." Requiring a permit for anything that isn’t structural or doesn’t directly impact the public (like a curb cut) seems, in my opinion, like a completely illegitimate racket.