Received better second job offer after accepting one already, please help

Hello all, please help me navigate through this awkward situation

I just graduated in December with my degree, the government shutdown slowed me up a bit but I finally got through several full rounds of interviews and received an offer for roughly 40k a year for a helpdesk position

Well, I accepted and went through the BG check and pre employment drug screen and everything and received a start date.

Today, I received a better job offer for about 10k more a year with benefits (the first job offer didn’t offer me any benefits)

What should I do in this situation? I want to accept the job that pays more obviously but this first company has already spent money on me and I feel kind of guilty about jumping ship, help?

usa jobs
usa jobs resume
usa hotel jobs
usajobs
usa jobs federal government
usa job in ksa
usa jobs
usa jobs login
usa jobs gov
usajobs.gov
www.usajobs.gov
usajobs.com
usajobs
usajobs.gov official