Here's the Explanation....
The fish, Greek ichthys (ixquj), is a symbol for Christ which has been in use since the days of the early church. In Greek, it is an acronym for Jesus Christ, the Son of God, our Savior:
Jesus - I h s o u V
Christ - C r i s t o V
God's - Q e o u
Son - U i o V
Savior - S w t h r
Frequently Christians were forced to worship secretly. The fish symbol served them well in difficult times because it generally would go unnoticed by a foe of Christianity when Christians used it to communicate. Placed outside a Christian's home, this symbol would announce silently that Christian Communion was to be observed secretly there that night. Artistic forms of the fish frequently decorated the Roman catacombs where Christians were forced to meet during persecution.
Notice that X was the first letter in the word that was Greek for Christ. In the fourth century, copies of the scriptures had to hand written. Commonly used terms were abbreviated using the letter abbreviation. The original four gospels were written in Greek and so the abbreviation of X for Christ remained.
So the next time you see the X used in Christmas instead of Christ, instead of being upset and feeling like it is an effort to take Christ out of Christmas, be reminded of early Christians trying to worship their Savior and later by people trying to speed up the process of making more copies of the scriptures.