[...deleted...]

Only the area of the triangle (here 6) matters, because any two triangles are equivalent under an area-preserving transformation of the plane, and such transformations take ellipses to ellipses (or circles which are a special case). So an equivalent question is: given an ellipse, what's the area of the smallest triangle circumscribing it? Again that depends only on the area of the ellipse, so we may as well take it to be a circle. Now the area of a triangle circumscribed about a given circle is proportional to its perimeter, which in turn is proportional to the sum of the cotangents of its half-angles. Since the sum of these half-angles is fixed at pi/2, and cot(x) is concave upwards on the interval 0<x<pi/2, the sum of cotangents is maximized when all the half-angles are equal, i.e. when the triangle is equilateral. The incircle of such a triangle occupies pi/sqrt(27) of its area (about 60.46%). The answer to the original question is thus 6*pi/sqrt(27)=2*pi/sqrt(3) or about 3.6276.

--Noam D. Elkies (elkies@zariski.harvard.edu)

Dept. of Mathematics, Harvard University