Why is wood burning fumes coming down to our lower fireplace,from our main floor fireplace ?

We live in a back slit house.We have one wood burning fireplace on the main floor and another on the bottom floor.Every time we have a fire in the top fireplace,it sends bad smoke smell to the lower one.Our dampers are operating correctly and both fireplaces are clean.Each fireplace has its own flue and are not connected.We have noticed that the 2 flues at the top of the fireplace chimney outside are level with each other.Can this be causing a problem of one sucking the fumes down another? If anyone knows the answer,please let us know.Thank you.