GLPN fine-tuned on NYUv2

Global-Local Path Networks (GLPN) model trained on NYUv2 for monocular depth estimation. It was introduced in the paper Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth by Kim et al. and first released in this repository.
Disclaimer: The team releasing GLPN did not write a model card for this model so this model card has been written by the Hugging Face team.

Model description

GLPN uses SegFormer as backbone and adds a lightweight head on top for depth estimation.

Intended uses & limitations

You can use the raw model for monocular depth estimation. See the model hub to look for
fine-tuned versions on a task that interests you.

How to use

Here is how to use this model:
from transformers import GLPNFeatureExtractor, GLPNForDepthEstimation<br /> import torch<br /> import numpy as np<br /> from PIL import Image<br /> import requests<br /> url = "http://images.cocodataset.org/val2017/000000039769.jpg"<br /> image = Image.open(requests.get(url, stream=True).raw)<br /> feature_extractor = GLPNFeatureExtractor.from_pretrained("vinvino02/glpn-nyu")<br /> model = GLPNForDepthEstimation.from_pretrained("vinvino02/glpn-nyu")<br /> # prepare image for the model<br /> inputs = feature_extractor(images=image, return_tensors="pt")<br /> with torch.no_grad():<br /> outputs = model(**inputs)<br /> predicted_depth = outputs.predicted_depth<br /> # interpolate to original size<br /> prediction = torch.nn.functional.interpolate(<br /> predicted_depth.unsqueeze(1),<br /> size=image.size[::-1],<br /> mode="bicubic",<br /> align_corners=False,<br /> )<br /> # visualize the prediction<br /> output = prediction.squeeze().cpu().numpy()<br /> formatted = (output * 255 / np.max(output)).astype("uint8")<br /> depth = Image.fromarray(formatted)<br />

For more code examples, we refer to the documentation.

BibTeX entry and citation info

@article{DBLP:journals/corr/abs-2201-07436,<br /> author = {Doyeon Kim and<br /> Woonghyun Ga and<br /> Pyunghwan Ahn and<br /> Donggyu Joo and<br /> Sehwan Chun and<br /> Junmo Kim},<br /> title = {Global-Local Path Networks for Monocular Depth Estimation with Vertical<br /> CutDepth},<br /> journal = {CoRR},<br /> volume = {abs/2201.07436},<br /> year = {2022},<br /> url = {https://arxiv.org/abs/2201.07436},<br /> eprinttype = {arXiv},<br /> eprint = {2201.07436},<br /> timestamp = {Fri, 21 Jan 2022 13:57:15 +0100},<br /> biburl = {https://dblp.org/rec/journals/corr/abs-2201-07436.bib},<br /> bibsource = {dblp computer science bibliography, https://dblp.org}<br /> }<br />