Why I Stopped Studying Mandarin(Elephant's Game, Part 1)

My best friend in middle school was from Taiwan. I spent most afternoons at his house where he taught me to play a game that he called Chinese Chess. 

My own Xiangqi board showing the
traditional characters on the red side
and universal glyphs on the black side.

In Mandarin the game is called Xiangqi, meaning "Elephant's Game." It evolved from an Indian board game that's also the ancestor of the chess played more widely in the rest of the world. My friend always won, in part because he'd grown up with this game and better grasped its strategy. But his advantage also came from the pieces being flat discs marked with Chinese characters, and one needs to discern each character to know which piece is which. 

My middle school friend was pretty good at teaching me: "You can remember that ⾺ is the Horse because of these four little brush strokes and a horse has four legs. And the bottom of 象 looks like an elephant's trunk." But the characters in Xiangqi are different for each side, and no matter how many times he went over which was what, I kept confusing them.

In high school I took a course in Mandarin. The first semester was easy: I'm adept at imitating accents and thrived off of Mandarin's nuanced consonants and tonal vowels, so learning spoken vocabulary and building that into sentences was fun and I found opportunities to speak my bits of Mandarin with people around the city. But when the second semester brought in Chinese's characters, I still had trouble discerning them because I just couldn't see them well enough to tell the more complicated characters apart. So I struggled, and at the end of that first year, I stopped studying Mandarin.

I still play Xiangqi, but made stickers for the pieces' non-Chinese symbols. And now I wonder: Do people who have low vision who grow up with Chinese struggle the same way I do in telling these characters apart? Or is my impediment more linguistic than visual?