Tag: paper
All the articles with the tag "paper".
The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
Published:This paper addresses a critical vulnerability in modern Large Language Models (LLMs): their susceptibility to prompt injection attacks, jailbreaks, and system prompt extractions. The authors argue that this stems from the lack of a clear instruction hierarchy, where LLMs treat instructions from application developers (system messages) with the same priority as those from potentially malicious users or third-party sources.
LLMs Can Teach Themselves to Better Predict the Future
Published:This paper introduces a novel framework for improving the forecasting capabilities of Large Language Models (LLMs) through outcome-driven fine-tuning. The method leverages model self-play to generate diverse reasoning trajectories and probabilistic forecasts for future events. These forecasts are then ranked based on their accuracy compared to actual outcomes, and the model is fine-tuned using Direct Preference Optimization (DPO). The results demonstrate significant accuracy improvements (7-10%) on Phi-4 14B and DeepSeek-R1 14B models, bringing their performance on par with much larger models like GPT-4o, without relying on human-curated reasoning samples. This approach has implications for decision-making across various sectors like finance, policy, and law.
Magma: A Foundation Model for Multimodal AI Agents
Published:Magma is a multimodal agentic AI model that can generate text based on the input text and image. The model is designed for research purposes and aimed at knowledge-sharing and accelerating research in multimodal AI, in particular the multimodal agentic AI. The main innovation of this model lies on the introduction of two technical innovations: Set-of-Mark and Trace-of-Mark, and the leverage of a large amount of unlabeled video data to learn the spatial-temporal grounding and planning.