1. The earth is closest to the sun in January and furthest in June.
Periapsis is the closest point, Apoasis is the furthest.
So the orbit is elliptical but only barely so, the furthest point is only ~3% more than the closest. Though that is still 3.1 million miles which is 400 Earths. You'd think that would matter for something, but nope. The closest point is much colder than the furthest for those of us in the Northern Hemisphere
2. The tilt is the main reason for seasonal differences.
The tilt of the earth is 23.5 degrees which at it's extreme accounts for a distance difference between Summer and Winter of at most 3200 mi at the poles (Assuming I did my trig correctly. )
So a straight distance of 3 million miles further from the sun doesn't matter but a tilt difference of 3200 miles apparently does. gees.
It turns out the reason for this is that the energy the sun transfers to us doesn't degrade much with that 3 million mile distance when it hits the earth directly ( i.e. around the equator ), but the tilt causes the energy towards the poles to get to be spread out and have less effect. Wikipedia gives us a good example of this.
So with a 30 degree tilt the energy is roughly 50% less compact meaning it's cooler per unit of area.
Now this made mathematical sense but it didn't make physical sense. How do you make energy less compact exactly? After thinking for a while I realized energy is conveyed as particles and by having one particle travel further ( due to the earth being at an angle ) than it's neighbor means the energy it imparts to the earth will be more spread out and less dense.
This made a bit of sense for say the US which is in the middle of the Northern Hemisphere, but what about the poles? Shouldn't they get a ton of heat in the summer and as much non heat in the winter? Such that you'd expect all the ice to completely melt in the summer?
3. The North Pole ( Arctic ) has weird days and warm waters.
Aside from the fact that the North Pole has 24 days in the summer and 24 nights in the winter, the key piece of info is that the sun is never directly above the pole, so it's always hitting the pole at an angle so the energy is always spread out. In addition the energy is in the form of radiation and snow and ice reflect around 70% of that away in the spring; though it does go down to 20% in the summer, however there's often cloud cover in the summer and that energy is still spread out and less 'effective.' So basically it's 24 hours days are just mediocre. The Arctic even has fairly warm oceans that never get much cooler than -2C and the sun still doesn't have a huge effect. (Wikipedia once again.)
4. The South Pole is very very cold.
Antarctica has much of the same properties though it is notoriously much much colder. So while the Arctic has large fluctuations in it's ice pack, the Antarctic doesn't. The South Pole gets a lot less love than the North Pole, so the best I can figure is that it is so much cooler due to a constant cold current (wikipedia) that circles the continent and keeps warm water away. This lets the ice persist which feeds back and causes the air to be cooler. I suppose if the North Pole wasn't surrounded by a lot of land it to might have a similar feature.
This pretty much satisfies my curiosity. There's a lot more going on at each level of this but digging through the layers goes a long way.