Build a translation app with LangChain

Prerequisites

Before starting this tutorial, you need:

• Python 3.11 or later installed on your computer. You can download Python from the official website.

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/python/foundry-local/langchain-integration

Install packages

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

pip install foundry-local-sdk-winml openai

pip install foundry-local-sdk openai

You also need to install the following LangChain package:

pip install langchain[openai]

Create a translation application

Create a new Python file named translation_app.py in your favorite IDE and add the following code:

from foundry_local_sdk import Configuration, FoundryLocalManager
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser


# Initialize the Foundry Local SDK
config = Configuration(app_name="foundry_local_samples")
FoundryLocalManager.initialize(config)
manager = FoundryLocalManager.instance

# Download and register all execution providers.
current_ep = ""


def _ep_progress(ep_name: str, percent: float):
    global current_ep
    if ep_name != current_ep:
        if current_ep:
            print()
        current_ep = ep_name
    print(f"\r  {ep_name:<30}  {percent:5.1f}%", end="", flush=True)


manager.download_and_register_eps(progress_callback=_ep_progress)
if current_ep:
    print()

# Load a model
model = manager.catalog.get_model("qwen2.5-0.5b")
model.download(
    lambda progress: print(
        f"\rDownloading model: {progress:.2f}%",
        end="",
        flush=True,
    )
)
print()
model.load()
print("Model loaded.")

# Start the web service to expose an OpenAI-compatible endpoint
manager.start_web_service()
base_url = f"{manager.urls[0]}/v1"

# Create a LangChain ChatOpenAI instance pointing to the local endpoint
llm = ChatOpenAI(
    base_url=base_url,
    api_key="none",
    model=model.id,
)

# Create a translation chain
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a translator. Translate the following text to {language}. Only output the translation, nothing else.",
        ),
        ("user", "{text}"),
    ]
)

chain = prompt | llm | StrOutputParser()

# Run the chain
result = chain.invoke({"language": "Spanish", "text": "Hello, how are you today?"})
print(f"Translation: {result}")

# Clean up
model.unload()
manager.stop_web_service()

References

• Reference: Foundry Local SDK reference
• Reference: Get started with Foundry Local

To run the application, open a terminal and navigate to the directory where you saved the translation_app.py file. Then, run the following command:

python translation_app.py

You're done when you see a Response: line with the translated text.

You should see output similar to:

Translating 'I love to code.' to French...
Response: <translated text>

Prerequisites

Before starting this tutorial, you need:

• Node.js 20 or later installed on your computer. You can download Node.js from the official website.

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/javascript/foundry-local/langchain-integration-example

Install packages

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

npm install foundry-local-sdk-winml openai

npm install foundry-local-sdk openai

Install LangChain packages

You also need to install the following Node.js packages:

npm install @langchain/openai @langchain/core

Create a translation application

Create a new JavaScript file named translation_app.js in your favorite IDE and add the following code:

import { ChatOpenAI } from "@langchain/openai";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { FoundryLocalManager } from 'foundry-local-sdk';

// Initialize the Foundry Local SDK
console.log('Initializing Foundry Local SDK...');

const endpointUrl = 'http://localhost:5764';

const manager = FoundryLocalManager.create({
    appName: 'foundry_local_samples',
    logLevel: 'info',
    webServiceUrls: endpointUrl
});
console.log('✓ SDK initialized successfully');

// Download and register all execution providers.
let currentEp = '';
await manager.downloadAndRegisterEps((epName, percent) => {
    if (epName !== currentEp) {
        if (currentEp !== '') process.stdout.write('\n');
        currentEp = epName;
    }
    process.stdout.write(`\r  ${epName.padEnd(30)}  ${percent.toFixed(1).padStart(5)}%`);
});
if (currentEp !== '') process.stdout.write('\n');

// Get the model object
const modelAlias = 'qwen2.5-0.5b'; // Using an available model from the list above
const model = await manager.catalog.getModel(modelAlias);

// Download the model
console.log(`\nDownloading model ${modelAlias}...`);
await model.download((progress) => {
    process.stdout.write(`\rDownloading... ${progress.toFixed(2)}%`);
});
console.log('\n✓ Model downloaded');

// Load the model
console.log(`\nLoading model ${modelAlias}...`);
await model.load();
console.log('✓ Model loaded');

// Start the web service
console.log('\nStarting web service...');
manager.startWebService();
console.log('✓ Web service started');


// Configure ChatOpenAI to use your locally-running model
const llm = new ChatOpenAI({
    model: model.id,
    configuration: {
        baseURL: endpointUrl + '/v1',
        apiKey: 'notneeded'
    },
    temperature: 0.6,
    streaming: false
});

// Create a translation prompt template
const prompt = ChatPromptTemplate.fromMessages([
    {
        role: "system",
        content: "You are a helpful assistant that translates {input_language} to {output_language}."
    },
    {
        role: "user",
        content: "{input}"
    }
]);

// Build a simple chain by connecting the prompt to the language model
const chain = prompt.pipe(llm);

const input = "I love to code.";
console.log(`Translating '${input}' to French...`);

// Run the chain with your inputs
await chain.invoke({
    input_language: "English",
    output_language: "French",
    input: input
}).then(aiMsg => {
    // Print the result content
    console.log(`Response: ${aiMsg.content}`);
}).catch(err => {
    console.error("Error:", err);
});

// Tidy up
console.log('Unloading model and stopping web service...');
await model.unload();
manager.stopWebService();
console.log(`✓ Model unloaded and web service stopped`);

#To run the application, open a terminal and navigate to the directory where you saved the translation_app.js file. Then, run the following command:

node translation_app.js

You're done when you see a Response: line with the translated text.

You should see output similar to:

Translating 'I love to code.' to French...
Response: J'aime le coder