Passive aggressive comment from coworker

I'm working on a project away from the office and the people I'm working with have been very polite and friendly towards me. A coworker called me up and asked me about some kind of technical issue with his computer so I helped him, and he asked me what I thought of the people I was working with. I replied that they were very nice people and he said "well, they may just be acting fake. Maybe they can't stand you and as soon as you leave the office, they talk badly about you." I told him that's possible, but I didn't think so. It just struck me as rude that I made a positive comment and he responded with such negativity, or am I being sensitive?

usa jobs
usa jobs resume
usa hotel jobs
usajobs
usa jobs federal government
usa job in ksa
usa jobs
usa jobs login
usa jobs gov
usajobs.gov
www.usajobs.gov
usajobs.com
usajobs
usajobs.gov official