1. Preview before you buy
Books, especially commentaries, are expensive. Try to preview them in some way before you buy them. This will save you money and disappointment.
2. Think about it
There are so many good books to buy. So many! Often times I've learned about a book and gotten really excited about a book and wanted to buy it only to have my initial excitement wear off a few days later. To combat this, I've instituted a waiting period before I buy anything to guard myself against impulse purchases.
3. You need more than commentaries
A good theological library should contain commentaries, bible dictionaries, introductory books covering portions of Scripture (NT, OT, Paul, the Gospels, the prophets, etc.) both of the historical critical type and the theological type, systematic theology texts, books of historical theology, lexicons and grammars, books on church history, and some basic references on the Apocrypha, Pseudepigrapha, and the Apostolic Fathers. You also should buy important monographs within each of these fields that provide in-depth studies on various matters.
4. Don't buy it just because...
it's cheap or it's by an author you like. I've seen the former mistake made often, especially related to commentaries. You have a lacuna in your library so when you see a book or series on sale that addresses your need you jump on it. Hold your horses, especially if you don't know much about the book/series you're buying.
5. Be diverse
No tradition gets everything right and there's great benefit to seeing Scripture from different angles. Especially when buying commentaries, try to buy books from several different positions.
6. Don't neglect older works
There are a lot of good works out there by dead guys. Read them!
7. Buy more than you think you need because you'll never know when you need it
I know that good reference books are expensive but you need to have enough books on your shelf that when you have a question about a particular passage or a significant topic you have a place or two to start looking for answers. You never know when that question will pop up, so it's best to be prepared in advance.
8. It's not just about you
When building your library, try to be other focused. Encourage people to borrow from you. Don't be afraid to give away a book that has been helpful to you.