Pranav Jeevan P

Research | Pranav Jeevan P

Professional Research

Sync Labs

As a Research Scientist at Sync Labs, I focus on building robust, efficient and state-of-the-art models for zero-shot video editing, lip-sync generation, and speech-driven facial animation. I work with flow-matching architectures, with a specific focus on optimizing inference efficiency, improving visual quality, and developing rigorous evaluation metrics for generative video. I also work on video super-resolution, TTS models, speech editing and language modeling.

Key Innovations & Highlights

🎬 Audio-Conditioned Lip-Syncing & Motion Transfer

Generative Synchronization

Developed industry-standard lip-sync systems using multimodal language models and Flow-Matching algorithms.

Motion Transfer & Extraction

Engineered robust pipelines for cross-video facial motion transfer, keybody extraction, and precise landmark tracking.

Speech & TTS Integration

Integrated conditioning into complex TTS models and interactive speech-editing workflows for pixel-perfect frame alignment.

📺 Next-Gen Video Super-Resolution (VSR) & Pixel Space Generation

Hybrid Generative VSR

Pioneered SOTA Video Super-Resolution models combining LLM-based Transformers, GANs, and Flow-Matching frameworks.

Pixel-Based Flow Matching

Investigated direct pixel-space flow matching architectures to bypass compression limits, eliminate VAE decoding artifacts, and unlock higher spatial fidelity for realistic video synthesis.

Reference-Guided Enhancement

Designed reference-frame-based VSR methods to preserve temporal consistency and fine texture details across frames.

Production-Grade Infrastructure

Built architectures engineered specifically for ultra-high-bitrate production formats (ProRes, EXR, etc.) and high-frame-rate master pipelines.

⚡ High-Efficiency & Low-Latency Inference

Advanced Distillation

Applied Advanced LoRA Distillation techniques to achieve drastically reduced step counts while maintaining visual fidelity.

Sampling & Latency Optimization

Utilized progressive flow matching, exploited sparsity, and developed specialized runtime techniques to minimize memory footprints and enable near-real-time generation.

Samsung Research

I worked with the AI Camera Team of Visual Intelligence Division at Samsung R&D Institute India, Bangalore (SRI-B), where I developed and optimized deep learning models for action recognition on edge devices. These models have been integrated into Samsung's flagship Galaxy S24 series in its Single Take feature.

Key Innovations & Highlights

Commercial Deployment

Using Single Video frames - Commercialized in Samsung's flagship Galaxy S24.

Patent Filed

Developed a Semi-Supervised framework with Progressive Pseudo-labeling for Animal Facial Action Recognition - Patent Filed (SRIB-20240130-007).

Tata Motors

I worked at the Application Engineering and also at Brake Department at the Engineering Research Center (ERC) of Tata Motors. I developed a comprehensive toolkit to analyse vehicle stability when subjected to various driving conditions and application components. I also developed an automated brake thermal analysis software for identifying potential brake issues during extreme driving scenarios.

Academic Research

PhD Thesis

Development of Resource-efficient architectures for computer vision such as image classification, semantic segmentation, image super-resolution, image inpainting and developing efficient evaluation metrics for generative models.

Award: Naik and Rastogi Award for Excellence in Ph.D. Research (2024–2026), awarded by IIT Bombay (formerly the Best PhD Thesis Award).

Key Innovations & Highlights

Wavelet-Based Architectures

Conceptualized and created WaveMix, WavePaint, and WaveMixSR, leveraging 2D-discrete wavelet transforms for superior performance in image classification, semantic segmentation, image super-resolution and image inpainting. These architectures deliver high performance with reduced computational requirements, outperforming state-of-the-art models.

Generative Evaluation Metrics

Designed normalizing flow-based evaluation metrics, FLD, FLD+, and RL2, which ensure monotonic behavior and deliver reliable estimates using significantly fewer real and synthetic images.

X-Former Architectures

Developed Hybrid X-Formers and Convolutional X-Formers, which integrate image-specific inductive biases, significantly improving performance with limited data and GPU resources.

M.Tech Thesis

Design and control of a lower extremity exoskeleton for rehabilitation.

Developed a functional 6-DOF active lower-extremity exoskeleton prototype and control pipeline designed to aid lower-body motor rehabilitation.

Key Innovations & Highlights

Biomechanical Mechanism Design

Designed an optimized 4-bar linkage mechanism for the hip joint (via 3-point analytical synthesis and Genetic Algorithms) and an offset revolute joint for the knee. This design accurately accounts for the shifting instantaneous center of rotation in biological joints.

Passive Gravity Balancing

Implemented zero-free-length spring balancing to make the exoskeleton's potential energy invariant across gait configurations, significantly reducing motor torque requirements during the swing phase.

Sensor Fusion & Gait State Estimation

Built a custom foot-pressure array using FlexiForce ground-reaction sensors. Trained an Artificial Neural Network (ANN) to map ground reaction forces directly to joint angular trajectories for adaptive, state-based motor control.

End-to-End Prototyping & Validation

Modeled and stress-tested (FEA) the structure in Autodesk Inventor, fabricated the custom aluminum frame via water-jet cutting, and experimentally validated real-time trajectory tracking on human subjects.

Recent Publications

Scroll through my recent research publications. Click on any paper to learn more.

Normalizing Flow Based Metric for Image Generation

2026

Quality Metrics Generative AI

ICML 2026

WaveMix: A Resource-efficient Neural Network for Image Analysis

2026

Token Mixer Efficiency

ICML 2026

Convolutional Xformers for Vision

2026

Hybrid Architecture Vision

ICML 2026

Spatially-Aware Mixture of Experts with Log-Logistic Survival Modeling for Whole-Slide Images

2026

Histopathology Survival Analysis

ISBI 2026

Survival Modeling from Whole Slide Images via Patch-Level Graph Clustering and Mixture Density Experts

2026

Histopathology Survival Analysis

ISBI 2026

WavePaint: Resource-efficient Token-mixer for Self-supervised Inpainting

2025

Inpainting Self-Supervised

ICCV 2025

FLD+: Data-efficient Evaluation Metric for Generative Models

2025

Quality Metrics Generative AI

ICCV 2025

Evaluation Metric for Quality Control and Generative Models in Histopathology Images

2025

Medical Imaging Histopathology

ISBI 2025

PathoGen-X: A Cross-Modal Genomic Feature Trans-Align Network for Enhanced Survival Prediction from Histopathology Images

2025

Histopathology Genomics

ISBI 2025

Which Backbone to Use: A Resource-efficient Domain Specific Comparison for Computer Vision

2025

Battle of Backbones Efficiency

TMLR

WaveMixSR-V2: Enhancing Super-resolution with Higher Efficiency

2025

Super Resolution Efficient AI

AAAI 2025

FLeNS: Federated Learning with Enhanced Nesterov-Newton Sketch

2024

2022

2024

Video Summarization Efficiency

WaveMixSR: A Resource-efficient Neural Network for Image Super-resolution

2024

Super Resolution Efficiency

WACV 2024

Advancing Gene Selection in Oncology: A Fusion of Deep Learning and Sparsity for Precision Gene Selection

2024

Genomics Cancer

Magnification Invariant Medical Image Analysis: A Comparison of Convolutional Networks, Vision Transformers, and Token Mixers

2024

Medical Imaging Histopathology

BIOSTEC 2024

Resource-Efficient Hybrid X-Formers for Vision

2022

Vision Transformers Efficiency

WACV 2022

“So You Think You’re Funny?”: Rating the Humour Quotient in Standup Comedy

2021

NLP Humor Detection

EMNLP 2021

Dr. Pranav Jeevan P

Professional Research

Sync Labs

Samsung Research

Tata Motors

Academic Research

PhD Thesis

M.Tech Thesis

Recent Publications

Normalizing Flow Based Metric for Image Generation

WaveMix: A Resource-efficient Neural Network for Image Analysis

Convolutional Xformers for Vision

Spatially-Aware Mixture of Experts with Log-Logistic Survival Modeling for Whole-Slide Images

Survival Modeling from Whole Slide Images via Patch-Level Graph Clustering and Mixture Density Experts

WavePaint: Resource-efficient Token-mixer for Self-supervised Inpainting

FLD+: Data-efficient Evaluation Metric for Generative Models

Evaluation Metric for Quality Control and Generative Models in Histopathology Images

PathoGen-X: A Cross-Modal Genomic Feature Trans-Align Network for Enhanced Survival Prediction from Histopathology Images

Which Backbone to Use: A Resource-efficient Domain Specific Comparison for Computer Vision

WaveMixSR-V2: Enhancing Super-resolution with Higher Efficiency

FLeNS: Federated Learning with Enhanced Nesterov-Newton Sketch

PawFACS: Leveraging Semi-Supervised Learning for Pet Facial Action Recognition

EDSNet: Efficient-DSNet for Video Summarization

WaveMixSR: A Resource-efficient Neural Network for Image Super-resolution

Advancing Gene Selection in Oncology: A Fusion of Deep Learning and Sparsity for Precision Gene Selection

Magnification Invariant Medical Image Analysis: A Comparison of Convolutional Networks, Vision Transformers, and Token Mixers

Resource-Efficient Hybrid X-Formers for Vision

“So You Think You’re Funny?”: Rating the Humour Quotient in Standup Comedy

Normalizing Flow Based Metric for Image Generation

WaveMix: A Resource-efficient Neural Network for Image Analysis

Convolutional Xformers for Vision

Spatially-Aware Mixture of Experts with Log-Logistic Survival Modeling for Whole-Slide Images

Survival Modeling from Whole Slide Images via Patch-Level Graph Clustering and Mixture Density Experts

WavePaint: Resource-efficient Token-mixer for Self-supervised Inpainting

FLD+: Data-efficient Evaluation Metric for Generative Models

Evaluation Metric for Quality Control and Generative Models in Histopathology Images

PathoGen-X: A Cross-Modal Genomic Feature Trans-Align Network for Enhanced Survival Prediction from Histopathology Images

Which Backbone to Use: A Resource-efficient Domain Specific Comparison for Computer Vision

WaveMixSR-V2: Enhancing Super-resolution with Higher Efficiency

FLeNS: Federated Learning with Enhanced Nesterov-Newton Sketch

PawFACS: Leveraging Semi-Supervised Learning for Pet Facial Action Recognition

EDSNet: Efficient-DSNet for Video Summarization

WaveMixSR: A Resource-efficient Neural Network for Image Super-resolution

Advancing Gene Selection in Oncology: A Fusion of Deep Learning and Sparsity for Precision Gene Selection

Magnification Invariant Medical Image Analysis: A Comparison of Convolutional Networks, Vision Transformers, and Token Mixers

Resource-Efficient Hybrid X-Formers for Vision

“So You Think You’re Funny?”: Rating the Humour Quotient in Standup Comedy