We are responsible to society for our donations?

Last week at the Leadership Summit, I was struck by a quick comment made by one of the speakers about our moral responsibility to society about how we choose to give our money as donations. Michael Porter suggested that since most of the time we are receiving a tax deduction for our charitable gifts, we are actually using society's money. In other words, a portion of our financial gift would have gone to the government/society if we had not chosen to give it somewhere else. The government in good faith is trusting that we would perhaps spend it better than they would? In any case, Porter suggests that because we are in a way using society's money, we should take interest in how our gifts are used. We're gambling with money that is not necessarily all ours. I thought that was an interesting thought.