The well-documented capacity-approaching performance of sparse codes in the realm of classical communications has inspired the search for their quantum counterparts. Sparse quantum codes are generally built as the amalgamation of two robust classical codes and are decoded via classical decoding algorithms. However, the quantum paradigm presents phenomena that act in a deleterious manner on sparse quantum codes when they are decoded based on classical methodologies. One such phenomenon is known as degeneracy, and it is a major contributor to why sparse quantum codes do not entirely evoke the stupendous error correcting abilities of their classical counterparts. In this paper, we adopt a group theoretical approach to discuss the issue of degeneracy as it relates to sparse quantum codes. Furthermore, we compare the decoding process of sparse quantum codes with that of sparse classical codes, illustrating the challenges that appear in the quantum domain. Finally, we provide a detailed example to illustrate the effects of degeneracy on sparse quantum codes and the challenges of designing an optimum decoder for these schemes.