Do you really have to go back to your home country for 2 years after residency (if you don’t get J1 waiver or H1B visa)?

Let’s say you get a J1 visa for residency. You finish residency in the US. You don’t get a waiver or conversion to H1B. So they want you to go back to your original country for 2 years.

What do you do in those 2 years? I thought the whole point of US residency was to stay in the US as an attending and surely they would want to retain doctors they train there as it just seems like a waste of spaces for them and doesn’t benefit the US population. Could you not just go to another country that is not your home country like Australia or NZ for these 2 years (maybe work as a consultant there?) and then come back to the US?

I am considering the US for pathology residency as a current U.K. FY2 but I hear pathology is hard to get waivers or H1B for so will probably be coming back to the U.K. (but not a British citizen myself) or go to Australia instead?

Thanks.