Meta’s SAM 3D - Creating 3D models from photos

Meta’s SAM 3D - Creating 3D models from photos

Last year I dabbled a bit with 3D models from images.

It was okay: low density, lots of noise, impressive but still too wonky for production use.

Recently, when I went down the SAM 3 Vision Model rabbit hole, I also found they have a SAM 3D.

SAM 3D looks like it takes it to a much higher polygon density than the model I played with last year.

Their playground examples all seem top-notch though on further investigation, if you step outside of their examples, the results are good, but not quite as perfect as they make it out to be.

They did release SAM3D open source, but unfortunately it is not part of SageMaker or Bedrock’s foundation models yet.

This is likely a big part of Meta’s plan to get the inside track on Augmented Reality.

It also could be used for robotics, as taking a 2D image and getting a rough idea of how it fits into real world 3D space would give the robot more context to interact with it.