Swatch Images Opening a Brand new page

Hello! Is there a way to make it so that when a customer clicks through a few different variation swatch images then clicks the back button they are brought back to the category page. Instead of being brought through all of the swatch images they clicked on prior?

For instance: if we go to this page https://swavellehospitality.com/product/abuja/?attribute_pa_real-color=silver and click on a couple of swatch images then click the back button on the top left corner we are brought back through all the swatch images we just clicked instead of back to the prior page.

Thank you!