This past summer at Trail of Bits was a season of inspiration, innovation, and growth thanks to the incredible contributions of our talented associates, who took on a diverse range of technical projects under the mentorship of Trail of Bits engineers. We’d like to delve into their accomplishments, from enhancing the efficiency of fuzzing tools and improving debugger performance to exploring the capabilities of deep learning frameworks.
Xiangan He: Scalable Circom determinacy checking with Circomference
Xiangan He’s work this summer was focused on building a tool to check for missing constraints and nondeterminacy in production-scale zero-knowledge (ZK) circuits. Existing security tools for these circuits were limited in their ability to handle circuits with more than 10 million constraints, prompting the development of Circomference. Inspired by tools like Picus and Ecne, Circomference uses easily swappable SMT solver backends orchestrated by a fast Rust orchestrator and determinacy propagator to scrutinize the determinacy of larger, more complex circuits commonly encountered in real-world scenarios.
Determinacy checking is crucial for identifying bugs within zero-knowledge circuits. Xiangan’s project demonstrated that tools like Circomference and Picus could detect vulnerabilities in 98.6% of a sample of 250 ZK circuits with known vulnerabilities. Moreover, due to improved memory usage and propagation heuristics, Circomference easily handles circuits that quickly cause Picus to run out of system RAM.
Circomference not only excels in efficiency but also effectively detects nondeterminacy in circuits used in real audits, making it invaluable for ensuring the integrity and security of zero-knowledge circuits.
Michael Lin: Fuzzing event tracing for Windows (ETW)
Michael Lin embarked on a project focused on fuzzing applications that consume events using Event Tracing for Windows (ETW). ETW plays a crucial role in Windows systems, serving various components and endpoint detection and response (EDR) solutions. However, since anyone can register a provider with the correct GUID, this process is vulnerable to exploitation.
Michael’s team began by selecting interesting EDRs and reverse engineering them to identify the providers they consumed events from. Since no existing testing or fuzzing frameworks matched the complexity of inter-process communication mechanisms like ETW, they had to develop their own.
The fuzzer they created aimed to generate random events sent to these providers with the goal of uncovering parsing bugs. It encountered intriguing challenges along the way, including difficulty in bypassing Windows process protection and in tracking fuzzing progress. Nonetheless, the team successfully automated much of the process and plans to apply the approach to other applications utilizing ETW.
Matheus Borella: Enhancing GDB and pwndbg
Matheus Borella’s summer project involved making improvements to GDB and pwndbg, a GDB plugin for reverse engineering and exploit development, with a particular focus on enhancing performance and adding features.
One remarkable achievement was a significant reduction in debugger startup times for users leveraging GDB Indexes. This change demonstrated a substantial speed improvement of up to 20 times during testing. Additionally, Matheus introduced features like adding __repr__ for certain Python types and sent patches (still to be merged) that extend the Python API with custom type creation and runtime symbol addition, enhancing GDB’s debugging and reverse engineering capabilities.
Their work also brought several quality-of-life improvements to pwndbg, including experimental use-after-free detection and new commands (plist, stepuntilasm, and break-if-[not-]taken). Along the way, they even discovered and fixed a bug in QEMU that had been causing GDB crashes in certain cases.
Patrick Dobranowski: Evaluating LLMs for security
Patrick Dobranowski’s project addressed the need to assess the effectiveness of large language models (LLMs) in various domains. Patrick’s project was to create a means to more easily determine which models are good at which tasks. During development, we also noticed existing metrics fell short in topics of interest to Trail of Bits, like Solidity language comprehension. Patrick then worked to create an evaluation framework, extended from HumanEval, to assess Solidity code comprehension.
Sanketh Menda: Empowering developers with ZKDocs
Sanketh Menda worked on addressing the gap between protocols described in cryptography research papers and implementations of the same protocols. In particular, they focused on zero-knowledge proofs and contributed content on the Inner Product Argument and its applications to polynomial commitment schemes to ZKDocs, distilling these protocols into their essential implementation details.
Sanketh also worked alongside the cryptography team on security assessments of zero-knowledge-related codebases, gaining hands-on experience in the field.
Kevin Chen: Investigating PyTorch for deep learning security
Kevin Chen’s project explored the correctness and security of PyTorch, a widely used Python framework for deep learning. While PyTorch is celebrated for its simplicity and efficiency, its intricate inner workings posed questions about correctness.
Kevin initially focused on PyTorch’s automatic differentiation engine, known as autograd, which is fundamental for neural network training. His meticulous study, leveraging dataflow analysis and debuggers, concluded that PyTorch developers adhere to critical rules. Kevin’s work uncovered insights into PyTorch’s code generation practices and identified potential areas for future research.
Sameed Ali: A fuzzer that actually follows directions!
In the realm of directed fuzzing, where tools use metrics like shortest-path-to-target(s) to discover specific code locations, Sameed’s work stands out. His project extended LibAFL to create a fuzzer that can genuinely follow directions and generates inputs that satisfy a sequence of preconditions.
Traditional reachability metrics often fall short in capturing the complexity of real-world bugs, as exploits often require a specific sequence of preconditions to be satisfied. Sameed’s innovative approach takes a sequence of targets and dynamically updates the shortest-path-to-target metric calculation as progress is made. This approach allows the fuzzer to generate inputs that hit more complex bugs, significantly advancing the state of the art in directed fuzzing.
Apply to our associate program!
The dedication and innovation of our associates underscore Trail of Bits’ commitment to advancing cybersecurity and technology. It was such a pleasure to work with the Summer Associate cohort this year, and we can’t wait to see what they accomplish next.
We’ll be opening up our summer associate application process in January next year!
Article Link: Summer associates 2023 recap | Trail of Bits Blog