Research
Find research shared by the Owlstown community.
Does learning the right latent variables necessarily improve in-context learning?
Abstract
Large autoregressive models like Transformers can solve tasks through in-context learning (ICL) without learning new weights, suggesting avenues for efficiently solving new tasks. For many tasks, e.g., linear regression, the data factor...
Publication · 6h
Learning Macro Variables with Auto-encoders
Abstract
Most causal variables that we reason over, in both science and everyday life, are coarse abstractions of low-level data. However, despite their importance, the field of causality lacks a precise theory of abstract “macro” variables and ...
Publication · 6h
Causal Representation Learning in Temporal Data via Single-Parent Decoding
Abstract
Scientific research often seeks to understand the causal structure underlying high-level variables in a system. For example, climate scientists study how phenomena, such as El Ni~no, affect other climate processes at remote locations ac...
Publication · 6h
In-Context Learning Can Re-learn Forbidden Tasks
Abstract
Despite significant investment into safety training, large language models (LLMs) deployed in the real world still suffer from numerous vulnerabilities. One perspective on LLM safety training is that it algorithmically forbids the model...
Publication · 6h
Evaluating Interventional Reasoning Capabilities of Large Language Models
Abstract
Numerous decision-making tasks require estimating causal effects under interventions on different parts of a system. As practitioners consider using large language models (LLMs) to automate decisions, studying their causal reasoning cap...
Publication · 6h
Demystifying amortized causal discovery with transformers
Abstract
Supervised learning approaches for causal discovery from observational data often achieve competitive performance despite seemingly avoiding explicit assumptions that traditional methods make for identifiability. In this work, we invest...
Publication · 6h
Adjusting Machine Learning Decisions for Equal Opportunity and Counterfactual Fairness
Abstract
Machine learning ( ml ) methods have the potential to automate high-stakes decisions, such as bail admissions or credit lending, by analyzing and learning from historical data. But these algorithmic decisions may be unfair: in learning ...
Publication · 6h
Sparsity regularization via tree-structured environments for disentangled representations
Abstract
Many causal systems such as biological processes in cells can only be observed indirectly via measurements, such as gene expression. Causal representation learning -- the task of correctly mapping low-level observations to latent causal...
Publication · 6h