LangChain Integration

LangChain is a powerful framework for building applications with large language models. OpenRouter seamlessly integrates with LangChain, giving you access to 100+ models for your chains, agents, and RAG applications.

Enhanced Capabilities

Combine LangChain's powerful abstractions with OpenRouter's model diversity for advanced AI applications.

Installation

Install LangChain and the OpenAI integration (which we'll configure for OpenRouter):

npm install langchain @langchain/openai
# or
yarn add langchain @langchain/openai
# or
pnpm add langchain @langchain/openai

Basic Setup

Configure LangChain to use OpenRouter's endpoint:

import { ChatOpenAI } from '@langchain/openai';

const model = new ChatOpenAI({
  modelName: 'anthropic/claude-3-opus',
  openAIApiKey: process.env.OPENROUTER_API_KEY,
  configuration: {
    baseURL: 'https://openrouter.ai/api/v1',
    defaultHeaders: {
      'HTTP-Referer': process.env.YOUR_SITE_URL,
      'X-Title': process.env.YOUR_APP_NAME,
    },
  },
  temperature: 0.7,
  maxTokens: 1000,
});

Chat Models

Use OpenRouter models with LangChain's chat interface:

Simple Chat

import { HumanMessage, SystemMessage } from '@langchain/core/messages';

const messages = [
  new SystemMessage('You are a helpful assistant.'),
  new HumanMessage('Explain quantum computing in simple terms.'),
];

const response = await model.invoke(messages);
console.log(response.content);

Streaming Responses

const stream = await model.stream([
  new HumanMessage('Write a short story about a time traveler.'),
]);

for await (const chunk of stream) {
  process.stdout.write(chunk.content);
}

Chains

Build complex chains with OpenRouter models:

Sequential Chain

import { PromptTemplate } from '@langchain/core/prompts';
import { LLMChain } from 'langchain/chains';
import { SequentialChain } from 'langchain/chains';

// First chain: Generate a topic
const topicTemplate = PromptTemplate.fromTemplate(
  'Generate an interesting topic about {subject}.'
);
const topicChain = new LLMChain({
  llm: model,
  prompt: topicTemplate,
  outputKey: 'topic',
});

// Second chain: Write an article
const articleTemplate = PromptTemplate.fromTemplate(
  'Write a detailed article about: {topic}'
);
const articleChain = new LLMChain({
  llm: model,
  prompt: articleTemplate,
  outputKey: 'article',
});

// Combine chains
const overallChain = new SequentialChain({
  chains: [topicChain, articleChain],
  inputVariables: ['subject'],
  outputVariables: ['topic', 'article'],
});

const result = await overallChain.call({
  subject: 'artificial intelligence',
});

console.log('Topic:', result.topic);
console.log('Article:', result.article);

Conversation Chain with Memory

import { ConversationChain } from 'langchain/chains';
import { BufferMemory } from 'langchain/memory';

const memory = new BufferMemory();
const conversationChain = new ConversationChain({
  llm: model,
  memory: memory,
});

// First interaction
const response1 = await conversationChain.call({
  input: "Hi! My name is Alice.",
});
console.log(response1.response);

// Second interaction (remembers context)
const response2 = await conversationChain.call({
  input: "What's my name?",
});
console.log(response2.response); // Should remember "Alice"

Agents

Create intelligent agents that can use tools and make decisions:

import { initializeAgentExecutorWithOptions } from 'langchain/agents';
import { Calculator } from 'langchain/tools/calculator';
import { WebBrowser } from 'langchain/tools/webbrowser';
import { ChatOpenAI } from '@langchain/openai';
import { Embeddings } from '@langchain/openai';

// Initialize embeddings for the web browser tool
const embeddings = new Embeddings({
  openAIApiKey: process.env.OPENROUTER_API_KEY,
  configuration: {
    baseURL: 'https://openrouter.ai/api/v1',
  },
});

// Create tools
const tools = [
  new Calculator(),
  new WebBrowser({ model, embeddings }),
];

// Create agent
const agent = await initializeAgentExecutorWithOptions(tools, model, {
  agentType: 'zero-shot-react-description',
  verbose: true,
});

// Use the agent
const result = await agent.call({
  input: 'What is the square root of the current year multiplied by 10?',
});

console.log(result.output);

RAG (Retrieval-Augmented Generation)

Build RAG applications with OpenRouter models:

import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter';
import { MemoryVectorStore } from 'langchain/vectorstores/memory';
import { OpenAIEmbeddings } from '@langchain/openai';
import { RetrievalQAChain } from 'langchain/chains';

// Initialize embeddings
const embeddings = new OpenAIEmbeddings({
  openAIApiKey: process.env.OPENROUTER_API_KEY,
  configuration: {
    baseURL: 'https://openrouter.ai/api/v1',
  },
});

// Load and split documents
const textSplitter = new RecursiveCharacterTextSplitter({
  chunkSize: 1000,
  chunkOverlap: 200,
});

const docs = await textSplitter.createDocuments([
  `OpenRouter provides a unified interface to access multiple AI models.
  It supports models from Anthropic, OpenAI, Google, Meta, and more.
  With automatic failover and intelligent routing, it ensures high availability.`,
]);

// Create vector store
const vectorStore = await MemoryVectorStore.fromDocuments(
  docs,
  embeddings
);

// Create retrieval chain
const chain = RetrievalQAChain.fromLLM(
  model,
  vectorStore.asRetriever()
);

// Query the chain
const response = await chain.call({
  query: 'What providers does OpenRouter support?',
});

console.log(response.text);

Model Selection Strategies

Leverage different models for different tasks:

// Create model instances for different use cases
const models = {
  // Fast, cost-effective for simple tasks
  fast: new ChatOpenAI({
    modelName: 'anthropic/claude-3-haiku',
    openAIApiKey: process.env.OPENROUTER_API_KEY,
    configuration: { baseURL: 'https://openrouter.ai/api/v1' },
  }),
  
  // Balanced for most tasks
  balanced: new ChatOpenAI({
    modelName: 'anthropic/claude-3-sonnet',
    openAIApiKey: process.env.OPENROUTER_API_KEY,
    configuration: { baseURL: 'https://openrouter.ai/api/v1' },
  }),
  
  // Maximum capability for complex tasks
  powerful: new ChatOpenAI({
    modelName: 'anthropic/claude-3-opus',
    openAIApiKey: process.env.OPENROUTER_API_KEY,
    configuration: { baseURL: 'https://openrouter.ai/api/v1' },
  }),
};

// Route based on task complexity
async function processTask(task: string, complexity: 'simple' | 'medium' | 'complex') {
  const model = complexity === 'simple' ? models.fast :
                complexity === 'medium' ? models.balanced :
                models.powerful;
  
  return await model.invoke([new HumanMessage(task)]);
}

Advanced Features

Custom Callbacks

import { BaseCallbackHandler } from '@langchain/core/callbacks/base';

class CustomCallbackHandler extends BaseCallbackHandler {
  name = 'CustomHandler';
  
  async handleLLMStart(llm: any, prompts: string[]) {
    console.log('Starting LLM call with prompts:', prompts);
  }
  
  async handleLLMEnd(output: any) {
    console.log('LLM call completed');
  }
  
  async handleLLMError(err: Error) {
    console.error('LLM error:', err);
  }
}

// Use with model
const modelWithCallbacks = new ChatOpenAI({
  modelName: 'openai/gpt-4-turbo',
  openAIApiKey: process.env.OPENROUTER_API_KEY,
  configuration: { baseURL: 'https://openrouter.ai/api/v1' },
  callbacks: [new CustomCallbackHandler()],
});

Output Parsers

import { StructuredOutputParser } from 'langchain/output_parsers';
import { z } from 'zod';

// Define output schema
const parser = StructuredOutputParser.fromZodSchema(
  z.object({
    title: z.string().describe('Article title'),
    summary: z.string().describe('Brief summary'),
    tags: z.array(z.string()).describe('Relevant tags'),
    sentiment: z.enum(['positive', 'neutral', 'negative']),
  })
);

// Create prompt with format instructions
const prompt = PromptTemplate.fromTemplate(
  `Analyze the following text and provide structured output.
{format_instructions}

Text: {text}`
);

const chain = prompt.pipe(model).pipe(parser);

const result = await chain.invoke({
  text: 'OpenRouter makes it easy to use multiple AI models...',
  format_instructions: parser.getFormatInstructions(),
});

console.log(result); // Typed object with title, summary, tags, sentiment

Error Handling & Retries

Implement robust error handling with automatic retries:

import { ChatOpenAI } from '@langchain/openai';

const robustModel = new ChatOpenAI({
  modelName: 'anthropic/claude-3-opus',
  openAIApiKey: process.env.OPENROUTER_API_KEY,
  configuration: { baseURL: 'https://openrouter.ai/api/v1' },
  maxRetries: 3,
  maxConcurrency: 5,
});

// With fallback models
async function callWithFallback(messages: any[]) {
  const models = [
    'anthropic/claude-3-opus',
    'openai/gpt-4-turbo',
    'google/gemini-pro',
  ];
  
  for (const modelName of models) {
    try {
      const model = new ChatOpenAI({
        modelName,
        openAIApiKey: process.env.OPENROUTER_API_KEY,
        configuration: { baseURL: 'https://openrouter.ai/api/v1' },
      });
      
      return await model.invoke(messages);
    } catch (error) {
      console.warn(`Model ${modelName} failed, trying next...`);
      if (modelName === models[models.length - 1]) {
        throw error;
      }
    }
  }
}

Performance Tip

Use streaming for better user experience and consider caching embeddings for frequently accessed documents in RAG applications.

Best Practices

Choose the right model for each task - don't use Opus for simple completions
Implement proper error handling and fallback strategies
Use streaming for long-form content generation
Cache embeddings and intermediate results when possible
Monitor token usage and costs through the OpenRouter dashboard
Use structured output parsers for reliable data extraction
Implement rate limiting for production applications

Example: Multi-Model RAG System

Here's a complete example of a production-ready RAG system using multiple models:

import { ChatOpenAI } from '@langchain/openai';
import { OpenAIEmbeddings } from '@langchain/openai';
import { MemoryVectorStore } from 'langchain/vectorstores/memory';
import { RetrievalQAChain } from 'langchain/chains';

class MultiModelRAG {
  private embeddings: OpenAIEmbeddings;
  private vectorStore: MemoryVectorStore;
  
  constructor() {
    this.embeddings = new OpenAIEmbeddings({
      openAIApiKey: process.env.OPENROUTER_API_KEY,
      configuration: { baseURL: 'https://openrouter.ai/api/v1' },
    });
  }
  
  async initialize(documents: string[]) {
    const textSplitter = new RecursiveCharacterTextSplitter({
      chunkSize: 1000,
      chunkOverlap: 200,
    });
    
    const docs = await textSplitter.createDocuments(documents);
    this.vectorStore = await MemoryVectorStore.fromDocuments(
      docs,
      this.embeddings
    );
  }
  
  async query(question: string, modelTier: 'fast' | 'balanced' | 'powerful' = 'balanced') {
    const modelMap = {
      fast: 'anthropic/claude-3-haiku',
      balanced: 'anthropic/claude-3-sonnet',
      powerful: 'anthropic/claude-3-opus',
    };
    
    const model = new ChatOpenAI({
      modelName: modelMap[modelTier],
      openAIApiKey: process.env.OPENROUTER_API_KEY,
      configuration: { baseURL: 'https://openrouter.ai/api/v1' },
      temperature: 0.3,
    });
    
    const chain = RetrievalQAChain.fromLLM(
      model,
      this.vectorStore.asRetriever({
        k: 4, // Return top 4 relevant chunks
      })
    );
    
    return await chain.call({ query: question });
  }
}

// Usage
const rag = new MultiModelRAG();
await rag.initialize([
  'Your document content here...',
  'More documents...',
]);

const answer = await rag.query(
  'What is the main topic of these documents?',
  'balanced'
);

Next Steps

Now that you've integrated LangChain with OpenRouter:

Explore the available models and their capabilities
Check out LangChain documentation for more advanced patterns
Learn about Vercel AI SDK integration for UI components
Monitor your usage and costs in the dashboard