Depending on the RV, you have either two separate inlets, one for city water and one to fill your freshwater tank OR one inlet with a valve that you turn for city water or fill.
So, first, make sure you are hooked up to the city inlet or you correctly have the valve set to city water. If that’s correct, then you likely have a backflow check valve stuck open, allowing water to flow back into your freshwater tank.
Here’s a diagram to help illustrate my point:
