My first shot would be like this:
1. Approximate Earth radius to say 6,360,000 meters.
2. Observe that with conical hexes, hex length will be proportional to
circumference of the sphere, which is the same as the circumference of a
circle of radius R. Circumference = Radius x 2 x pi. So hex length is
proportional to altitude above the center of the earth.
3. So hex length (L) = altitdue above sea level (ASL) + 6,360,000 meters /
6,360,000 meters.
4. So at 0 ASL. L = 1 standard hex length. At 80,000m ASL, L = (6,360,000
+ 80,000) / 6,360,000, = 1.0126 x standard hex length.
At that point, I think I'd decide it was close enough and not worry about
adding hexes as I ascended.