Well the real answer is, we don't know. The gospels tell us that it was in a rented room, but we aren't given the exact address. Nonetheless, there is a "traditional site" in Jerusalem that is identified as the "upper room" where Jesus ate the Last Supper and then, later on Pentecost, the Holy Spirit came upon the early church.
I have been in the so-called upper room several times and I am not impressed. The room is located over another suspect site, the so-called Tomb of David. Neither is old enough to be the location they are identified with. Nonetheless, tour guides still take groups there. I was once in the room when a group of Pentecostals visited the site. You can imagine the kinds of things that were going on in the room. What disappointed me most, however, was not their actions but that the tour guide led them to believe this was the actual spot.
It seems that the site is also the source of a feud between the Vatican and the Israeli authorities. The Vatican wants control, but Israel has run the site since 1967. NPR ran a short piece on the upper room and the dispute over who will control it. You can hear it here.