Research

How accurately and consistently do LLM agents report their beliefs about an interactive environment, and what can these reports reveal about task success?

• Developed an LLM-assisted pipeline that grounds proposition templates in agent trajectories and assigns True / False / Unknown reference labels using formal logical rules and Bayesian inference.
• Evaluated World, Epistemic, Affordance, and Decision propositions in ALFWorld, WebShop, and Noisy Cabinets, measuring accuracy alongside permutation, negation, sequential, and rephrasing consistency.
• Found that Affordance and Decision accuracy is positively associated with task success after centering within each environment, when both reference labels and self-reports are True or False.
• Showed that Unknown-response tendencies contribute to cross-model consistency differences, and developed a response-tendency model linking accuracy and negation consistency to explain their decline along longer interaction histories.

Can directions that accurately decode a model's internal state also steer its decisions when that state changes?

• Constructed controlled tasks from ALFWorld and WebShop trajectories that combine a source input's state with a target input's decision rule and answer mapping, distinguishing counterfactual recomputation from answer or source-plan copying.
• Learned and validated a reference steering subspace to establish that the required answer change is achievable, then compared probe-derived interventions at the same layer and position with fixed sample pairs and replacement operation.
• Across four models and both environments, found near-perfect linear probing accuracy in the orthogonal complement of the reference subspace, while direct probe-derived steering achieved near-zero counterfactual recomposition accuracy.
• Tested covariance-transformed directions and combinations of multiple probe directions; their gains in the full activation space did not carry over to the complement, showing that high probing accuracy alone is insufficient evidence of effective steering.

When should supervised fine-tuning stop so that the following reinforcement learning stage reaches the best final performance on mathematical reasoning?

• Performed SFT and RL post-training on multiple sizes of Qwen2.5 models using the OpenR1 dataset, continuously saving checkpoints throughout SFT.
• Ran RL separately from each checkpoint and found that, before the validation-loss minimum, later checkpoints begin RL with lower entropy but higher reward; response length consistently follows a decrease-then-increase pattern.
• Found that RL reward continues to improve after SFT validation loss reaches its minimum and early overfitting begins, declining only after validation loss rises substantially; this places the optimal SFT stopping point inside the early-overfitting regime.
• Observed the pattern clearly on 3B and 7B models; the 0.5B model was too small to learn the OpenR1 mathematical problems effectively.

A multi-slot auction for mixed item and advertisement rankings, where content contrast creates ordering externalities in click-through rates.

• Designed an evaluator architecture to capture ordering externalities and recover the underlying click-through rate.
• Designed a generator architecture that produces the best sequence from item features and bidding prices.
• Trained both networks and derived supporting theoretical results.