Jonathan Koch

Projects & Research

My Work

A selection of professional and research projects. None of these are coursework. For smaller projects and experiments, see my GitHub repositories.

Industry

Research

RPAL: Advanced Object Manipulation with the Barrett Hand

Research Scientist · USF Robot Perception and Action Lab

Research focused on enhancing robotic object manipulation through advanced representation learning. The goal: achieve precise control in tasks such as controlled object dropping using the Barrett Hand robot.

Multi-Objective Representation Learning

Encoder: State Representation

Encoder Architecture

Transformer-based architecture processing tactile data, images, torque measurements, and joint angles. Trained with temporal, spatial, value, and state reconstruction objectives to create a linear learned representation.

Dynamix: Forward Dynamics Prediction

Dynamix Model

Predicts future states from current state embeddings and actions via a Delta Network and Predictor, enabling multi-step trajectory prediction in embedding space.

Critiq: Inverse Dynamics & Reward Prediction

Critiq Model

Estimates actions and rewards from state transitions. Complements Dynamix through inverse dynamics learning with a time estimator for temporal context.

The shared encoder ensures consistent state representation across forward and inverse dynamics, creating a versatile foundation for advanced manipulation tasks.

Teach-a-Bull (IEEE AI Group)

Project Lead · USF IEEE

An extension of LLMaAiT-BE: formalizing graph-based generations over long-form text documents. The system uses LLMs in complex state-based environments modeled as MDPs, producing higher-quality generation than one-shot prompting for books, lectures, papers, and more.

Also proposed a distribution recreation hypothesis for assessing generation quality: if the generated document is semantically contained within the span of expert-generated documents, the generation is considered successful.

Personal & Open Source

MicrogradPlus

Auto-differentiation computational graph library extended from Andrej Karpathy's Micrograd. Extends scalar differentiation to support vector operations in a PyTorch-like style with n-dimensional tensors. Validated against PyTorch's auto-differentiation.

Reinforcement Learning in Chess

Applying reinforcement learning to chess using Monte Carlo Tree Search to approximate game state values. A neural network learns to emulate simulation values, with increasing reliance on state value predictions as the model improves. Inspired by DeepMind's AlphaGo and AlphaZero.

Gwen — Virtual Desktop Assistant

A voice-activated virtual assistant built in Python that controls and manages desktop environments. Features automated speech responses, wake-word activation, and integration with media platforms (Spotify, Netflix, YouTube). Includes an LLM backend server for conversation and chain-of-thought reasoning.

Robbie the DanciBULL Robot

USF HackaBull 2023

Built in 24 hours: a simulated robot that dances to any song. Uses policy and feature extraction networks to map audio data and joint angles to a probability distribution over 17 joints. Data pipeline constructed from Just Dance videos using a joint-angle extraction algorithm for offline RL, refined with RL-HF.

CoderSchoolAI

Open-source reinforcement learning library designed to make Agent AI accessible. Built in Python with PyTorch integration, providing neural network blocks, game environments, agent abstract classes, RL algorithms (DQN, PPO), and replay buffers. Used in both educational contexts and personal robotics research.

TerriBULL Robotics: BullBot

Robotics platform connecting the V5 Brain microcontroller with NVIDIA Jetson Nano for serial data communication and real-time object detection. Architecture includes a task manager for autonomous actions, mechanical system abstraction layers, and a decision model for path planning. ML research focused on offline and online RL (DDPG+HER) in simulated environments.

Astaria — 2D Action RPG

A 2D Action RPG built in the Godot engine featuring arcade mode with enemy waves, health potion drops, weapon upgrade systems, and hand-crafted pixel art. A solo project exploring game development and design.