> ## Documentation Index
> Fetch the complete documentation index at: https://docs.memsync.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Development

> Development setup and testing with MemSync

## Development Environment Setup

This guide covers setting up MemSync for local development, testing, and contributing to the project.

<Info>
  **Prerequisites**: Python 3.9+, PostgreSQL, and Redis are required for local development.
</Info>

## Local Installation

### Step 1: Clone the Repository

```bash theme={null}
git clone https://github.com/memsync/memsync.git
cd memsync
```

### Step 2: Install Dependencies

MemSync uses Poetry for dependency management:

<CodeGroup>
  ```bash Install with Poetry theme={null}
  poetry install
  ```

  ```bash Install All Dependencies theme={null}
  poetry install
  poetry run pip install groq together boto3 litellm ollama chromadb weaviate weaviate-client sentence_transformers vertexai \
                      google-generativeai elasticsearch opensearch-py vecs pinecone pinecone-text faiss-cpu langchain-community \
                      upstash-vector azure-search-documents
  ```
</CodeGroup>

### Step 3: Environment Configuration

Create a `.env` file in the project root:

```env theme={null}
# Database Configuration
POSTGRES_HOST=localhost
POSTGRES_PORT=5432
POSTGRES_DB=memsync_dev
POSTGRES_USER=postgres
POSTGRES_PASSWORD=your_password

# Redis Configuration  
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB=0

# API Keys
OPENAI_API_KEY=your_openai_key
API_KEY=your_api_key

# Supabase (for authentication)
SUPABASE_URL=your_supabase_url
SUPABASE_KEY=your_supabase_key
```

### Step 4: Database Setup

Initialize the PostgreSQL database with pgvector extension:

```bash theme={null}
# Create database and install pgvector
createdb memsync_dev
psql memsync_dev -c "CREATE EXTENSION vector;"

# Run migrations
cd server
alembic upgrade head
```

### Step 5: Start Development Server

<CodeGroup>
  ```bash Start API Server theme={null}
  cd server
  python main.py
  ```

  ```bash With Docker Compose theme={null}
  docker-compose up -d
  ```
</CodeGroup>

The API server will be available at `http://localhost:8000`

## Testing

MemSync includes comprehensive test suites for all components:

### Running Tests

```bash theme={null}
# Run all tests
poetry run pytest

# Run specific test file
poetry run pytest tests/test_memory.py

# Run with coverage
poetry run pytest --cov=memsync tests/
```

### Test Categories

<AccordionGroup>
  <Accordion title="Unit Tests">
    Test individual components like memory extraction, embedding generation, and vector operations.

    ```bash theme={null}
    pytest tests/test_memory.py
    pytest tests/test_embeddings.py
    pytest tests/test_user_registry.py
    ```
  </Accordion>

  <Accordion title="Integration Tests">
    Test API endpoints and database interactions.

    ```bash theme={null}
    pytest tests/test_api_key.py
    pytest tests/test_buffer.py
    ```
  </Accordion>

  <Accordion title="Memory Buffer Tests">
    Test the buffered memory system for conversation processing.

    ```bash theme={null}
    pytest tests/test_buffer.py
    ```
  </Accordion>
</AccordionGroup>

### Integration Testing

Test external integrations with sample data:

```bash theme={null}
# Test Reddit integration
python tests/integration_tester.py --type reddit --username sample_user

# Test memory extraction
python tests/memory_tester.py --input "sample conversation"
```

## Development Tools

### Code Quality

MemSync uses several tools to maintain code quality:

<CodeGroup>
  ```bash Formatting theme={null}
  # Format code with ruff
  ruff format .

  # Run all formatting tools
  make format
  ```

  ```bash Linting theme={null}
  # Check code with ruff
  ruff check .

  # Run pre-commit hooks
  pre-commit run --all-files
  ```

  ```bash Type Checking theme={null}
  # Run mypy type checking
  mypy memsync/
  ```
</CodeGroup>

### Available Make Commands

```bash theme={null}
# Install dependencies
make install

# Run tests
make test

# Format and lint code
make format

# Build package
make build

# Generate documentation
make docs
```

## Performance Testing

### Benchmarking

MemSync includes comprehensive benchmarking against the Locomo dataset:

```bash theme={null}
# Run memory indexing benchmark
python evaluation/run_experiments.py \
   --technique_type memsync \
   --method add \
   --input_file evaluation/dataset/locomo10.json

# Run search benchmark  
python evaluation/run_experiments.py \
   --technique_type memsync \
   --method search \
   --input_file evaluation/dataset/locomo10.json

# Evaluate results
python evaluation/evals.py \
   --input_file evaluation/results/memsync_search_results.json
```

### Performance Metrics

Key metrics tracked:

* **Memory extraction accuracy**: How well facts are extracted from conversations
* **Search relevance**: Quality of semantic search results
* **Storage efficiency**: Memory usage vs. retrieval accuracy
* **Processing speed**: Time to index and search memories

## API Development

### Testing API Endpoints

Use the interactive API documentation at `http://localhost:8000/docs` or test with curl:

```bash theme={null}
# Health check
curl http://localhost:8000/healthcheck

# Store memories (requires auth)
curl -X POST "http://localhost:8000/v1/memories" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Test message"}], "agent_id": "test", "thread_id": "test", "source": "test"}'

# Search memories
curl -X POST "http://localhost:8000/v1/memories/search" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "test query", "limit": 5}'
```

### Adding New Endpoints

1. Define request/response models in `server/main.py`
2. Implement the endpoint function
3. Add tests in `tests/`
4. Update API documentation

Example endpoint structure:

```python theme={null}
@app.post("/v1/new-endpoint")
async def new_endpoint(request: NewEndpointRequest, auth: CurrentAuth):
    # Validate request
    # Process with MemSync
    # Return response
    pass
```

## Configuration Options

### Memory Configuration

Customize memory extraction and storage:

```python theme={null}
config = {
    "llm": {
        "provider": "openai",  # or "gemini"
        "config": {
            "model": "gpt-4o",
            "temperature": 0.0
        }
    },
    "embedder": {
        "provider": "openai",  # or custom endpoint
        "config": {
            "model": "text-embedding-3-large",
            "embedding_dims": 1024
        }
    },
    "vector_store": {
        "provider": "pgvector",
        "config": {
            "collection_name": "memories_v5",
            "embedding_model_dims": 1024
        }
    },
    "memory_categories": ["career", "interests", "relationships", ...]
}
```

### Integration Configuration

Configure external service integrations:

```python theme={null}
# Social media integrations
REDDIT_CLIENT_ID = "your_reddit_client_id"
REDDIT_CLIENT_SECRET = "your_reddit_client_secret"

# Document processing
DOCLING_API_KEY = "your_docling_key"

# Monitoring
DATADOG_API_KEY = "your_datadog_key"
```

## Debugging

### Common Issues

<AccordionGroup>
  <Accordion title="Database Connection Errors">
    Ensure PostgreSQL is running and pgvector extension is installed:

    ```bash theme={null}
    # Check PostgreSQL status
    pg_isready

    # Install pgvector if missing
    psql -d memsync_dev -c "CREATE EXTENSION IF NOT EXISTS vector;"
    ```
  </Accordion>

  <Accordion title="Memory Extraction Not Working">
    Check LLM configuration and API keys:

    ```bash theme={null}
    # Test OpenAI connection
    python -c "import openai; print(openai.api_key)"

    # Check memory extraction logs
    tail -f logs/memsync.log
    ```
  </Accordion>

  <Accordion title="Vector Search Issues">
    Verify embeddings are being generated correctly:

    ```bash theme={null}
    # Test embedding generation
    python tests/test_embeddings.py

    # Check vector store connection
    python operations/check_embeddings.py
    ```
  </Accordion>
</AccordionGroup>

### Logging

Enable detailed logging for debugging:

```python theme={null}
import logging
logging.basicConfig(level=logging.DEBUG)

# Or set environment variable
export LOG_LEVEL=DEBUG
```

## Contributing

### Development Workflow

1. **Fork and clone** the repository
2. **Create a feature branch**: `git checkout -b feature/your-feature`
3. **Make changes** and add tests
4. **Run tests**: `pytest`
5. **Run linting**: `ruff check .`
6. **Commit changes**: Follow conventional commit format
7. **Push and create PR**

### Code Standards

* Follow PEP 8 style guidelines
* Add type hints for all functions
* Write tests for new functionality
* Update documentation as needed
* Use conventional commit messages

### Testing Requirements

All PRs must include:

* Unit tests for new functionality
* Integration tests for API changes
* Performance benchmarks for core features
* Documentation updates

## Deployment

### Production Setup

1. **Environment variables**: Configure all required environment variables
2. **Database**: Set up PostgreSQL with pgvector in production
3. **Redis**: Configure Redis for task queue
4. **Monitoring**: Set up Datadog or similar monitoring
5. **Security**: Configure CORS, rate limiting, and authentication

### Docker Deployment

```bash theme={null}
# Build production image
docker build -f server/Dockerfile -t memsync:latest .

# Run with docker-compose
docker-compose -f docker-compose.prod.yml up -d
```

For more detailed deployment instructions, see the [deployment guide](https://github.com/memsync/memsync/wiki/Deployment).

## Need Help?

* **Documentation**: Check our [API Reference](/api-reference/introduction)
* **Issues**: Report bugs on [GitHub Issues](https://github.com/memsync/memsync/issues)
* **Community**: Join our [Discord community](https://discord.gg/memsync)
* **Email**: Contact us at [dev@memsync.ai](mailto:dev@memsync.ai)