my server will only return the message to the client that sends a message to the server.
How will the server get the addresses of the other clients?
If the server does have the addresses of the others, say in a table, then it will need to connect to them. They will each have to be servers to THE server who would be acting like a client. Ie the clients will wait for THE server to connect to them and send them a message.
I'd think you'd want it so that each client connects to the server(logins) and then can receive messages from the server. A client after logging in would send a message to the server that was to be sent to other clients and the server would have connections to those clients and be able to send them the message.