I have this exact same issue with "open to internet"
The period shortly after 3 or so people can join with no issues, but if they disconnect they cannot reconnect, and others cannot connect some period of time after hosting.
Unfortunately I don't know whats causing the issue.
We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.