The pilot flew four skydiving flights without refueling. On the last flight, after the skydivers exited the airplane, the pilot initiated a descent and the airplane experienced a total loss of engine power. The pilot knew he could not make it back to the airport and made a forced landing to a gravel area.