Depending on your needs there is one other alternative… barycentric coordinates are the area of opposite triangles over the area of the sum of the triangles.
You might want do draw a picture of this… you’ve got triangle T, made of points abc. You have a point inside, p. The triangle A is pbc. The triangle B is pca. The triangle C is pab.
The position of P is Area(A)/Area(T) * a + Area(B)/Area(T) * b + Area©/Area(T) * c.
However, a natural extension of this when the point isn’t in T is to replace Area(T) in the above equation with Area(A)+Area(B)+Area©. It’s just replacing the normalizing term.
So, the behavior of this is a little bit different than the other solution with projection. The projection solution will project to the plane, and allow inside/outside tests. This solution assumes that you want every point in 3D to map to some point inside the triangle, and may or may not be useful for your application. Meh, probably not, but I think it’s kind of interesting…