Mesh Generation with Transformers 

[Type: Master Thesis, Focus: Machine Learning, Computer Graphics]​

​

The objective of this project is to investigate the potential of transformer-based models for 3D mesh generation (link). Specifically, the project explores how discrete or structured representations of meshes - such as vertices, faces, or learned latent codes - can be modeled as sequences and generated using GPT-style transformers. By encoding mesh geometry and topology into a sequence-friendly representation, the problem of mesh generation can be reframed as an autoregressive modeling task, similar to language modeling. A transformer is then trained to learn the distribution over these mesh tokens, enabling the sequential generation of new 3D meshes. This approach leverages the strong sequence modeling capabilities of transformers to capture both local geometric details and global structural consistency in meshes. The project aims to provide insights into how transformer architectures can be effectively applied to 3D geometry, advancing the state of generative modeling for mesh-based representations and enabling the synthesis of coherent and high-quality 3D shapes.

​

Image Generation Using GPT-Based Model and VQ-VAE

[Type: Bachelor Thesis, Focus: Machine Learning]

​

The objective of this project is to investigate the potential of combining a transformer-based model, such as GPT, with a Vector Quantized Variational Autoencoder (VQ-VAE) for the purpose of image generation. VQ-VAE is a type of generative model that effectively compresses images into discrete latent representations, making them suitable for sequence modeling.In this project, we explore how the discrete latent codes produced by a trained VQ-VAE can be treated similarly to tokens in a language model. By using a GPT-style transformer to model the distribution over these latent codes, we aim to generate new images in a sequential, token-by-token manner. This approach leverages the powerful generative capabilities of transformers in the domain of visual data, allowing for the creation of coherent and high-quality images. Ultimately, the project aims to provide insights into the synergy between VQ-VAE and transformer-based models, demonstrating how they can be integrated to push the boundaries of generative image modeling.

​

Monocular Fire Reconstruction

[Type: Master Thesis, Focus: Computer Vision, Machine Learning]

​

Traditional reconstructions of fire are based on a set of inward-facing cameras. The large baselines enable precise triangulation of the individual flames. This, however, comes with the main disadvantage of cost and intense calibration. For this reason, this project aims to explore the possibility of significantly reducing the amount of required cameras by introducing powerful pre-trained methods, such as monocular depth, point tracking, and diffusion-based novel-view-synthesis. While approaching the target of using only a single camera, photometric losses as well the generalization capabilities of the reconstructed model will be evaluated.

​

Learning based reverse engineering of 3D file formats

[Type: Bachelor or Master Thesis, Focus: Machine Learning]

​

This project aims to develop a neural-network architecture that receives a binary 3D file of unknown format as input and outputs a file format specification. To train this network, a method to create synthetic data should be developed. On one hand, arbitrary file format specifications can be synthesized and on the other hand 3D files can either be created procedurally or by using a dataset of 3D objects which are then augmented and converted to the previously synthesized file format specifications.​

​​​​​