Integrations

Connect MemSync with external data sources and services

Integrations

MemSync can import and process data from various external sources to enrich user profiles and memories. These integrations allow you to build comprehensive user context from existing digital footprints.

Available Integrations

Social Media

Import posts, interactions, and profile data from social platforms

Documents

Process PDFs, Word docs, and other file formats

Chat History

Import conversation history from messaging platforms

Social Media Integrations

LinkedIn

Import professional information, posts, and network data from LinkedIn profiles.

import requests

# Index LinkedIn profile data
response = requests.post("https://api.memchat.io/v1/integrations/index/social",
    headers={"Authorization": "Bearer YOUR_TOKEN"},
    json={
        "type": "LINKEDIN",
        "username": "john-doe-engineer",
        "agent_id": "my-chatbot",
        "thread_id": "integration-001",
        "config": {
            "include_posts": True,
            "include_connections": False,
            "max_posts": 50
        }
    }
)

print(f"Task ID: {response.json()['task_id']}")

What gets imported:

• Professional experience and education
• Skills and endorsements
• Published posts and articles
• Professional summary and bio
• Certifications and achievements

Twitter/X

Import tweets, interactions, and profile information.

# Index Twitter profile data
response = requests.post("https://api.memchat.io/v1/integrations/index/social",
    headers={"Authorization": "Bearer YOUR_TOKEN"},
    json={
        "type": "TWITTER",
        "username": "johndoe_dev",
        "agent_id": "my-chatbot", 
        "thread_id": "integration-002",
        "config": {
            "include_tweets": True,
            "include_replies": False,
            "max_tweets": 100
        }
    }
)

What gets imported:

• Recent tweets and retweets
• Bio and profile information
• Interests inferred from activity
• Communication style and tone
• Topics of discussion

Reddit

Import posts, comments, and subreddit activity.

# Index Reddit activity
response = requests.post("https://api.memchat.io/v1/integrations/index/social",
    headers={"Authorization": "Bearer YOUR_TOKEN"},
    json={
        "type": "REDDIT",
        "username": "johndoe_reddit",
        "agent_id": "my-chatbot",
        "thread_id": "integration-003",
        "config": {
            "include_posts": True,
            "include_comments": True,
            "max_items": 200
        }
    }
)

What gets imported:

• Posts and comments across subreddits
• Interests and hobbies from subreddit activity
• Opinions and preferences
• Communication patterns
• Areas of expertise or interest

Document Integration

File Upload and Processing

Process various document formats to extract meaningful information.

# Index document content
response = requests.post("https://api.memchat.io/v1/integrations/index/file",
    headers={"Authorization": "Bearer YOUR_TOKEN"},
    json={
        "type": "FILE",
        "url": "https://example.com/resume.pdf",
        "agent_id": "my-chatbot",
        "thread_id": "integration-004",
        "config": {
            "extract_metadata": True,
            "chunk_size": 1000,
            "process_images": False
        }
    }
)

Supported formats:

• PDF documents
• Word documents (.docx, .doc)
• Plain text files
• Markdown files
• PowerPoint presentations
• Excel spreadsheets

What gets extracted:

• Key facts and information
• Professional experience (from resumes)
• Educational background
• Skills and certifications
• Project descriptions
• Personal notes and insights

Document Processing Examples

Resume Processing

Extract career information from CV/resume documents

# Process resume document
resume_integration = {
    "type": "FILE",
    "url": "https://storage.example.com/resume.pdf",
    "config": {
        "document_type": "resume",
        "extract_contact_info": True,
        "extract_skills": True,
        "extract_experience": True
    }
}

Extracted memories:

• Work experience and job titles
• Educational qualifications
• Technical and soft skills
• Certifications and achievements

Personal Notes

Process personal documents and journals

# Process personal notes
notes_integration = {
    "type": "FILE", 
    "url": "https://storage.example.com/personal-notes.docx",
    "config": {
        "document_type": "personal",
        "extract_goals": True,
        "extract_interests": True,
        "privacy_level": "high"
    }
}

Extracted memories:

• Personal goals and aspirations
• Interests and hobbies
• Thoughts and reflections
• Plans and projects

Academic Papers

Extract research interests and expertise areas

# Process academic work
academic_integration = {
    "type": "FILE",
    "url": "https://storage.example.com/research-paper.pdf", 
    "config": {
        "document_type": "academic",
        "extract_keywords": True,
        "extract_methodology": True
    }
}

Extracted memories:

• Research interests and specializations
• Academic achievements
• Technical expertise areas
• Methodological preferences

Chat History Integration

ChatGPT History

Import conversation history from ChatGPT to understand user interaction patterns.

# Index ChatGPT conversation history
response = requests.post("https://api.memchat.io/v1/integrations/index/chat",
    headers={"Authorization": "Bearer YOUR_TOKEN"},
    json={
        "type": "CHATGPT",
        "url": "https://storage.example.com/chatgpt-conversations.json",
        "agent_id": "my-chatbot",
        "thread_id": "integration-005",
        "config": {
            "filter_by_date": "2024-01-01",
            "include_system_messages": False,
            "max_conversations": 50
        }
    }
)

What gets imported:

• Conversation topics and themes
• Questions and interests
• Problem-solving patterns
• Communication preferences
• Learning goals and objectives

Custom Chat Platforms

You can also integrate with custom chat platforms by formatting the data appropriately:

# Format custom chat data
chat_data = {
    "conversations": [
        {
            "id": "conv_001",
            "messages": [
                {"role": "user", "content": "I'm working on a React project"},
                {"role": "assistant", "content": "That's great! What kind of React project?"}
            ],
            "timestamp": "2024-03-20T10:00:00Z"
        }
    ]
}

# Upload and process
response = requests.post("https://api.memchat.io/v1/integrations/index/chat",
    headers={"Authorization": "Bearer YOUR_TOKEN"},
    json={
        "type": "CHATGPT",  # Use CHATGPT type for custom formatted data
        "data": chat_data,
        "agent_id": "my-chatbot",
        "thread_id": "integration-006"
    }
)

Integration Workflow

Asynchronous Processing

All integrations are processed asynchronously to handle large datasets efficiently:

1. Submit Integration Request

   Send a request to the appropriate integration endpoint

2. Receive Task ID

   Get a task ID to track the processing status

3. Monitor Progress

   Use the task ID to check processing status

4. Access Results

   Once complete, memories are automatically added to the user’s profile

Checking Integration Status

# Check integration status
def check_integration_status(task_id):
    response = requests.get(f"https://api.memchat.io/v1/integrations/status/{task_id}",
        headers={"Authorization": "Bearer YOUR_TOKEN"}
    )
    
    status = response.json()
    print(f"Status: {status['status']}")
    
    if status['status'] == 'SUCCESS':
        print(f"Processed items: {status['result']['indexed_items']}")
        print(f"Processing time: {status['result']['processing_time']}")
    elif status['status'] == 'FAILED':
        print(f"Error: {status['error_message']}")
    
    return status

Status Responses

{
  "task_id": "task_123",
  "status": "SUCCESS",
  "result": {
    "indexed_items": 150,
    "processing_time": "2.5s",
    "memories_created": 45,
    "categories_found": ["career", "interests", "learning"]
  },
  "error_message": null
}

Integration Configuration

Common Configuration Options

Content Filtering

Control what content gets processed

config = {
    "date_range": {
        "start": "2024-01-01",
        "end": "2024-12-31"
    },
    "content_types": ["posts", "comments"],
    "exclude_retweets": True,
    "min_content_length": 50
}

Privacy Settings

Control how sensitive information is handled

config = {
    "privacy_level": "high",  # high, medium, low
    "exclude_private_info": True,
    "anonymize_names": False,
    "exclude_sensitive_categories": ["finance", "health"]
}

Processing Options

Customize how content is processed

config = {
    "chunk_size": 1000,  # For document processing
    "extract_metadata": True,
    "language": "en",
    "summarize_long_content": True
}

Rate Limiting

Control processing speed to respect API limits

config = {
    "rate_limit": 10,  # requests per second
    "batch_size": 50,
    "retry_on_failure": True,
    "max_retries": 3
}

Best Practices

Data Quality

Source Selection

Choose high-quality, relevant data sources

• Recent data is generally more valuable than old data
• Professional platforms (LinkedIn) for career information
• Social platforms for interests and personality
• Personal documents for goals and aspirations

Content Filtering

Filter out low-quality or irrelevant content

# Good filtering practices
config = {
    "min_content_length": 50,  # Skip very short posts
    "exclude_spam": True,
    "language": "en",  # Focus on user's primary language
    "relevance_threshold": 0.7
}

Privacy Considerations

Respect user privacy and data sensitivity

• Always get explicit user consent
• Exclude sensitive personal information
• Allow users to review imported data
• Provide data deletion options

Performance Optimization

Batch Processing

Process data in appropriate batch sizes

# Optimal batch sizes by source type
batch_sizes = {
    "twitter": 100,
    "linkedin": 50, 
    "reddit": 200,
    "documents": 10
}

Incremental Updates

Process only new data for regular updates

# Track last integration date
last_integration = get_last_integration_date(user_id, "twitter")

config = {
    "since_date": last_integration,
    "incremental": True
}

Error Handling

Implement robust error handling for integrations

def robust_integration(integration_request):
    try:
        response = requests.post("/v1/integrations/index/social", 
            json=integration_request, timeout=30)
        return response.json()
    except requests.exceptions.Timeout:
        return {"error": "Integration timed out, try smaller batch"}
    except requests.exceptions.HTTPError as e:
        return {"error": f"HTTP error: {e.response.status_code}"}

Security and Privacy

Authentication

Integrations require proper authentication:

# Using API key 
headers = {
    "X-API-Key": "YOUR_API_KEY",
    "Content-Type": "application/json"
}

Data Handling

• Encryption: All data is encrypted in transit and at rest
• Access Control: Only authorized users can access integration data
• Audit Logs: All integration activities are logged for security
• Data Retention: Users can control how long integration data is stored

Compliance

MemSync integrations comply with:

• GDPR: Right to access, modify, and delete personal data
• CCPA: California Consumer Privacy Act requirements
• SOC 2: Security and privacy controls
• Platform ToS: Respect for third-party platform terms of service

Common Use Cases

Onboarding New Users

def onboard_user_with_integrations(user_id, integration_preferences):
    """Quickly build user profile from external sources"""
    
    integrations = []
    
    # Professional background
    if integration_preferences.get('linkedin'):
        linkedin_task = start_linkedin_integration(
            username=integration_preferences['linkedin_username']
        )
        integrations.append(linkedin_task)
    
    # Personal interests
    if integration_preferences.get('reddit'):
        reddit_task = start_reddit_integration(
            username=integration_preferences['reddit_username']
        )
        integrations.append(reddit_task)
    
    # Monitor all integrations
    return monitor_integrations(integrations)

Profile Enrichment

def enrich_existing_profile(user_id):
    """Enhance existing profile with additional data sources"""
    
    current_profile = get_user_profile(user_id)
    
    # Identify gaps in profile
    missing_categories = identify_missing_categories(current_profile)
    
    # Suggest relevant integrations
    suggestions = []
    if 'career' in missing_categories:
        suggestions.append("Consider importing LinkedIn data")
    if 'interests' in missing_categories:
        suggestions.append("Consider importing social media data")
    
    return suggestions

Data Migration

def migrate_from_other_platform(user_id, platform_data):
    """Migrate user data from another platform"""
    
    # Format data for MemSync
    formatted_conversations = format_conversations(platform_data)
    
    # Import as chat history
    task = start_chat_integration(
        data=formatted_conversations,
        user_id=user_id
    )
    
    return task

Troubleshooting

Common Issues

Integration Failures

Possible causes:

• Invalid username or profile not found
• Rate limiting from source platform
• Authentication issues
• Network connectivity problems

Solutions:

• Verify usernames and profile accessibility
• Implement retry logic with exponential backoff
• Check authentication credentials
• Monitor API rate limits

Low Quality Results

Possible causes:

• Insufficient source data
• Poor data quality in source
• Incorrect configuration

Solutions:

• Increase data volume if available
• Filter out low-quality content
• Adjust processing parameters
• Try different time ranges

Processing Timeouts

Possible causes:

• Large dataset size
• Complex document processing
• System load

Solutions:

• Reduce batch sizes
• Split large integrations into smaller chunks
• Retry failed integrations
• Contact support for assistance

Next Steps

Authentication

Learn about authentication methods for integrations

Memory Types

Understand how integrated data becomes memories

User Profiles

See how integrations enrich user profiles

Integration APIs

Explore the complete integration API documentation