Orphanages seem to be a real problem, but this is what I have managed to find so far.
(1) For a long time, orphanages were based on faith, so there were Catholic, Protestant and specific Protestant orphanages. Even if the state was responsible for removal of children (as in my father's case) the child was usually put into an orphanage of the family's faith. Hence my dad ended up in a Catholic boy's orphanage in Minneapolis, and his sisters in a Catholic orphanage in St. Paul. I found their cousins in a protestant orphanage also in St. Paul. (No idea why the cousins should have been in an orphanage.) I know that in Goodhue County, the Lutheran Church at Vasa also ran an orphanage.
(2) Records are (or tend to be) held by which ever organization "inherited" the management of the orphanage. Orphanages could have changed names and been managed by a variety of organizations in the course of their history. Thus the fact that an orphanage of a particular name no longer exists doesn't mean the records are lost, or that they have been put in, say, a county archive. In the case of my dad's cousins, the orphanage is long gone, but the Minnesota Historical Society was able to tell me the names of the various organizations which ran the orphanage in the period since 1920, and this enabled me to get in touch with the organization currently holding the records. (Unfortunately, their archivist hasn't been very forthcoming with help, however.)
3) If you can do a search on a census with an every name index, you may be able to find the child you are looking for in the institution if they were there during the census year. (That's how I found my dad, his sisters, and his cousins.)
4) Catholic orphanage records are mostly held by the Archdiocese in St. Paul, but so far I have no experience of trying to deal with them to get records.
After the 1960s, it became increasingly common to try to get children fostered rather than sent to an institution. I guess county courts may have records of this, or perhaps the social services of a county. Whether they will impart these is another matter.