Factored Cognition

We are doing research on Factored Cognition, led by Brian and Benjamin Sturgeon, this research explores using causal influence diagrams to explain and mitigate risk scenarios. When chaining parallel and sequential calls to large language models, you can create a causal graph that can be used to debug agent architectures and ask questions about intent alignment for AGI.Here are some ways that factored cognition (FC) can be used for AI safety use cases:- Identifying and mitigating risks- Ensuring compliance with safety regulations- Building trustworthy AI systems: FC can be used to ensure that AI systems are transparent and explainable.- Help policy makers visually understand the potential risks and benefits of AI.

Large language models (LLMs) are becoming ubiquitous and are often used to answer difficult questions that have important ethical and moral dimensions. However, most LLMs are trained with a unidimensional ethical framework imparted by its designers. To begin to remedy this problem, we employ factored cognition to augment the interpretability of the model’s ethical and moral reasoning. In this paper, we demonstrate our API which takes in a question and breaks it into subquestions that prompt the model for expansions on the problem that explore a wider moral space. The answers to the subquestions are then collected and compiled into a more interpretable response that better illustrates the process by which the model arrived at an answer. We benchmark our approach to establish that model performance is slightly decreased, but mostly left intact compared to the standalone model in moral benchmarks.Paper Morally Factored Cognition ⤴

Factored cognition represents an appealing architectural framework for increasingly capable artificial intelligence. Decomposing system faculty into loosely coupled, specialized components promises to provide benefits like heterogeneity and independent enhancement without interfering with overall behavioral prediction. However, such modularity also poses inherent verification challenges for reliability and interpretability at scale. In this paper, we investigate applying formal methods based on mechanized reasoning to improve rigor and safety for factored language architectures. Specifically, we leverage the Lean 3 interactive theorem prover to mathematically prove useful semantic and composability properties for word embedding and vocabulary modules under a factored design. We formalize metric interpretability specifications by which to verify constituent embedding spaces, alongside basic lemmas and theorems proving composability of representations given interface assumptions on the modules. Our work demonstrates principles by which to provide transparency, trust, and predictability to factored models and implement those principles to discharge non-trivial proof obligations related to semantic clustering and vocabulary composition. Thus formal methods have the potential to tackle integrative challenges that arise under factored cognition, while retaining the benefits of modular specialization across intelligent system faculties.