Thanks for the help guys.

I think I settled on constructing the matrix like this:


And the schematic is:



Let me know if you see any problems. Also, the book I am learning from recommended using the74LS174 decoder. and has provided example code.