Well finally figured the solution for it and stating the same here, Just in case, if others are also looking for an answer.

There are couple of ways to run  multiple instances of several Odoo/OpenUpgrade servers:

1. Docker

2. Virtual Environment

3. Servers running on different port.

I followed option 3, which I found it to be relatively simple, all you need to do is just specify different XML_port, and redirect the config file to different locations.

By default Odoo server will use the config file from Home directory (in Ubuntu), hence by specifying a different path to use the config file, solved the problem.