Tag: prompt-engineering

All the articles with the tag "prompt-engineering".

• The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions

  Published:Mar 15, 2025

  This paper addresses a critical vulnerability in modern Large Language Models (LLMs): their susceptibility to prompt injection attacks, jailbreaks, and system prompt extractions. The authors argue that this stems from the lack of a clear instruction hierarchy, where LLMs treat instructions from application developers (system messages) with the same priority as those from potentially malicious users or third-party sources.

• Retrieval Augmented Generation or Long-Context LLMs

  Published:Feb 22, 2025

  This document summarizes the findings of a comprehensive study comparing Retrieval Augmented Generation (RAG) and Long-Context (LC) Large Language Models (LLMs) for processing lengthy contexts. The study benchmarks both approaches across various public datasets using recent LLMs (Gemini-1.5-Pro, GPT-4O, and GPT-3.5-Turbo). The key finding is that LC models, when resourced sufficiently, generally outperform RAG in terms of average performance. However, RAG maintains a significant cost advantage due to the reduced input length to the LLM. Based on these observations, the study introduces SELF-ROUTE, a method that intelligently routes queries to either RAG or LC based on model self-reflection, significantly reducing computational costs while maintaining performance comparable to LC. The findings provide guidance for building long-context applications utilizing both RAG and LC.

• PromptWizard: The future of prompt optimization through feedback-driven self-evolving prompts

  Published:Jan 29, 2025

  This document reviews the key concepts and findings from two sources related to PromptWizard, a prompt optimization framework developed by Microsoft Research. These sources highlight the limitations of existing prompt optimization techniques, particularly for closed-source Large Language Models (LLMs), and introduce PromptWizard as a novel, iterative approach that leverages feedback and iterative refinement.