# Migration from ExaBGP 3.x to 4.x

**Complete guide for upgrading from ExaBGP 3.x to 4.x**

> ⚠️ **CRITICAL: You MUST Upgrade from 3.x**
>
> **ExaBGP 3.x uses Python 2, which is deprecated and no longer supported.** Python 2 reached end-of-life on January 1, 2020 and receives no security updates.
>
> **All users must upgrade to ExaBGP 4.x or 5.0.0** for security and compatibility.

> ⚠️ **Breaking Changes**: ExaBGP 3.x → 4.x includes **breaking changes** requiring configuration and code updates.

> ℹ️ **About ExaBGP 5.0.0**: If you're considering upgrading beyond 4.x, note that **ExaBGP 5.0.0 introduces additional breaking changes** from 4.x (Python 3.8+, JSON API changes). See [4.x → 5.0.0 Migration Guide](From-4.x-to-5.x) for details. Most users should upgrade to 4.x first, then decide if 5.0.0 is needed.

---

## Table of Contents

- [Overview](#overview)
- [Should You Upgrade?](#should-you-upgrade)
- [What Changed](#what-changed)
- [Configuration Syntax Changes](#configuration-syntax-changes)
- [JSON Format Changes](#json-format-changes)
- [Command-Line Changes](#command-line-changes)
- [API Changes](#api-changes)
- [Migration Steps](#migration-steps)
- [Testing Your Migration](#testing-your-migration)
- [Rollback Plan](#rollback-plan)
- [Common Issues](#common-issues)

---

## Overview

**Release**: ExaBGP 4.0.0 (~2017)

**Severity**: 🟡 **MODERATE** - Configuration and some API changes required

**Timeline**: ExaBGP 3.x is **deprecated and no longer maintained**. All users should upgrade to 4.x.

---

## Should You Upgrade?

### ✅ YES - You MUST Upgrade

**ExaBGP 3.x is built on Python 2, which is:**
- ❌ **End-of-life since January 1, 2020**
- ❌ **No security patches or updates**
- ❌ **Removed from most modern Linux distributions**
- ❌ **Incompatible with modern Python packages**

**All ExaBGP 3.x users must upgrade immediately for:**
- ✅ Security updates and patches
- ✅ Python 3 support (Python 2 is dead)
- ✅ New features (FlowSpec improvements, EVPN, BGP-LS, etc.)
- ✅ Better performance and stability
- ✅ Modern BGP features (ADD-PATH, extended next-hop, etc.)
- ✅ Continued maintenance and support

### Plan Your Migration:

⚠️ Configuration files need manual updates
⚠️ JSON output format changed (if using JSON parser)
⚠️ Some command-line arguments changed
⚠️ Test thoroughly before production deployment

---

## What Changed

### Breaking Changes

| Area | Change | Impact |
|------|--------|--------|
| **Configuration** | Syntax changes | 🔴 **HIGH** - Manual updates required |
| **JSON Format** | Output structure changed | 🟡 **MEDIUM** - Parser updates needed |
| **CLI Arguments** | Some flags changed | 🟡 **MEDIUM** - Script updates needed |
| **API Commands** | Minor syntax changes | 🟢 **LOW** - Most commands compatible |

### New Features in 4.x

- ✅ Full FlowSpec support (RFC 5575)
- ✅ EVPN support (RFC 7432)
- ✅ BGP-LS support (RFC 7752)
- ✅ ADD-PATH support (RFC 7911)
- ✅ Extended next-hop (RFC 5549)
- ✅ Python 3 support
- ✅ Better error messages
- ✅ Improved performance

---

## Configuration Syntax Changes

### Major Configuration Changes

The configuration file syntax changed significantly between 3.x and 4.x.

### Example: Basic Neighbor Configuration

**ExaBGP 3.x:**
```ini
neighbor 192.168.1.1 {
    router-id 192.168.1.2;
    local-address 192.168.1.2;
    local-as 65001;
    peer-as 65000;

    # 3.x syntax
    capability {
        route-refresh;
        graceful-restart 120;
    }
}
```

**ExaBGP 4.x:**
```ini
neighbor 192.168.1.1 {
    router-id 192.168.1.2;
    local-address 192.168.1.2;
    local-as 65001;
    peer-as 65000;

    # 4.x syntax - similar but some keywords changed
    capability {
        route-refresh enable;
        graceful-restart 120;
    }
}
```

### Process Configuration Changes

**ExaBGP 3.x:**
```ini
process announce {
    run /usr/local/bin/announce.py;
}

neighbor 192.168.1.1 {
    # ... neighbor config ...
    process announce;
}
```

**ExaBGP 4.x:**
```ini
process announce {
    run /usr/local/bin/announce.py;
    encoder text;  # New: explicit encoder
}

neighbor 192.168.1.1 {
    # ... neighbor config ...

    api {
        processes [ announce ];  # New: api block
    }
}
```

### Family Configuration Changes

**ExaBGP 3.x:**
```ini
neighbor 192.168.1.1 {
    # ... neighbor config ...

    family {
        ipv4 unicast;
        ipv4 flow;
    }
}
```

**ExaBGP 4.x:**
```ini
neighbor 192.168.1.1 {
    # ... neighbor config ...

    family {
        ipv4 unicast;
        ipv4 flow;
    }
}
```

**Note**: Family syntax is similar, but some address family names changed.

---

## JSON Format Changes

### JSON Output Structure Changed

If you use `encoder json` to receive BGP updates, the JSON format changed significantly.

### Example: Route Announcement

**ExaBGP 3.x JSON:**
```json
{
  "type": "update",
  "neighbor": "192.168.1.1",
  "announce": {
    "ipv4 unicast": {
      "100.10.0.0/24": {
        "next-hop": "192.168.1.2"
      }
    }
  }
}
```

**ExaBGP 4.x JSON:**
```json
{
  "exabgp": "4.2.25",
  "time": 1699564800.0,
  "type": "update",
  "neighbor": {
    "address": {
      "local": "192.168.1.2",
      "peer": "192.168.1.1"
    },
    "asn": {
      "local": 65001,
      "peer": 65000
    },
    "message": {
      "update": {
        "announce": {
          "ipv4 unicast": {
            "100.10.0.0/24": [
              {
                "next-hop": "192.168.1.2"
              }
            ]
          }
        }
      }
    }
  }
}
```

**Changes:**
- More structured (nested `neighbor` object)
- Added metadata (`exabgp`, `time`, `asn`)
- Route attributes now in arrays
- More detailed peer information

**Impact**: Programs parsing JSON output need updates.

---

## Command-Line Changes

### CLI Argument Changes

Some command-line arguments changed between 3.x and 4.x.

**ExaBGP 3.x:**
```bash
# 3.x command line
env exabgp.log.level=DEBUG exabgp /etc/exabgp/exabgp.conf
exabgp --help
```

**ExaBGP 4.x:**
```bash
# 4.x command line (mostly compatible)
exabgp /etc/exabgp/exabgp.conf
exabgp --help
exabgp version
```

**Environment Variables** (mostly compatible):
```bash
# Both 3.x and 4.x
export exabgp.daemon.user=exabgp
export exabgp.log.level=INFO
export exabgp.log.destination=stdout

# 4.x added
export exabgp.api.ack=true
```

---

## API Changes

### API Command Compatibility

**Most API commands are compatible** between 3.x and 4.x.

### Commands That Work in Both Versions

```python
# These work identically in 3.x and 4.x
announce route 100.10.0.0/24 next-hop self
withdraw route 100.10.0.0/24
announce flow route { match { destination 100.10.0.0/24; } then { discard; } }
```

### New Commands in 4.x

```python
# New in 4.x
announce attributes next-hop self nlri 100.10.0.0/24 100.20.0.0/24  # Bulk announcements
clear adj-rib out
flush adj-rib out
```

### ACK Feature (4.x Only)

**ExaBGP 3.x:**
- No ACK responses
- Fire-and-forget commands
- No way to know if command succeeded

**ExaBGP 4.x:**
- ACK enabled by default
- Sends `done`, `error`, `shutdown` responses
- Programs can verify command success

**Migration Note**: If your 3.x programs don't read STDIN, they will hang in 4.x unless you:
- **Option 1**: Update programs to read ACK responses (recommended)
- **Option 2**: Disable ACK (choose based on ExaBGP version):
  - Environment variable: `export exabgp.api.ack=false` (4.x and 5.x)
  - Runtime command: Send `disable-ack` or `silence-ack` (5.x/main only)

See [ACK Feature Documentation](API-Overview#command-acknowledgment-ack-feature) for details.

---

## Migration Steps

### Step 1: Backup Everything

```bash
# Backup configuration
cp /etc/exabgp/exabgp.conf /etc/exabgp/exabgp.conf.3x.backup

# Backup API programs
tar czf /tmp/exabgp-api-programs-backup.tar.gz /usr/local/bin/exabgp-*

# Document current version
exabgp version > /tmp/exabgp-version-3x.txt
```

---

### Step 2: Install ExaBGP 4.x (Test Environment)

**Option A: Using pip (recommended):**
```bash
# Install ExaBGP 4.x
pip install exabgp

# Verify version
exabgp version
# Should show: ExaBGP 4.2.x
```

**Option B: From source:**
```bash
git clone https://github.com/Exa-Networks/exabgp.git
cd exabgp
git checkout 4.2  # Use latest 4.2.x tag
pip install .
```

---

### Step 3: Update Configuration File

**Automated conversion** (if available):
```bash
# No official converter exists - manual updates required
```

**Manual updates**:
1. Read through your 3.x configuration
2. Check configuration syntax in [Configuration Syntax Guide](Configuration-Syntax)
3. Update process blocks to use `api { }` structure
4. Add `encoder` directive to process blocks
5. Verify family statements

**Example conversion:**

**Before (3.x):**
```ini
process announce {
    run /usr/local/bin/announce.py;
}

neighbor 192.168.1.1 {
    router-id 192.168.1.2;
    local-address 192.168.1.2;
    local-as 65001;
    peer-as 65000;

    family {
        ipv4 unicast;
    }

    process announce;
}
```

**After (4.x):**
```ini
process announce {
    run /usr/local/bin/announce.py;
    encoder text;  # Added
}

neighbor 192.168.1.1 {
    router-id 192.168.1.2;
    local-address 192.168.1.2;
    local-as 65001;
    peer-as 65000;

    family {
        ipv4 unicast;
    }

    api {  # Changed
        processes [ announce ];
    }
}
```

---

### Step 4: Test Configuration Syntax

```bash
# Test configuration parsing (ExaBGP 4.x)
exabgp --test /etc/exabgp/exabgp.conf

# Note: In ExaBGP 5.x+, use: exabgp configuration validate /etc/exabgp/exabgp.conf

# If errors occur, check:
# - Process block syntax (api { } structure)
# - encoder directives
# - Family statements
```

---

### Step 5: Update API Programs (If Using JSON)

If you use `encoder json` and parse JSON output:

**Update JSON parser:**
```python
# 3.x parser
def parse_3x(line):
    msg = json.loads(line)
    if msg['type'] == 'update':
        routes = msg['announce']['ipv4 unicast']
        # ...

# 4.x parser
def parse_4x(line):
    msg = json.loads(line)
    if msg['type'] == 'update':
        routes = msg['neighbor']['message']['update']['announce']['ipv4 unicast']
        # ...
```

---

### Step 6: Handle ACK Feature

**If your programs don't read STDIN:**

**Option 1 - Disable ACK (simpler, but no error feedback):**
```bash
# Environment variable (works on 4.x and 5.x)
export exabgp.api.ack=false
exabgp /etc/exabgp/exabgp.conf

# OR runtime command (5.x/main only)
# Send: disable-ack or silence-ack
```

**Option 2 - Update programs to read ACK (recommended):**
```python
#!/usr/bin/env python3
import sys
import select
import time

def wait_for_ack(expected_count=1, timeout=30):
    """
    Wait for ACK responses with polling loop (new in 4.x).
    ExaBGP may not respond immediately, so we poll with sleep.
    Handles both text and JSON encoder formats.
    """
    import json
    received = 0
    start_time = time.time()

    while received < expected_count:
        if time.time() - start_time >= timeout:
            return False

        ready, _, _ = select.select([sys.stdin], [], [], 0.1)
        if ready:
            line = sys.stdin.readline().strip()

            # Parse response (could be text or JSON)
            answer = None
            if line.startswith('{'):
                try:
                    data = json.loads(line)
                    answer = data.get('answer')
                except:
                    pass
            else:
                answer = line

            if answer == "done":
                received += 1
            elif answer == "error":
                return False
            elif answer == "shutdown":
                raise SystemExit(0)
        else:
            time.sleep(0.1)

    return True

# Send command
sys.stdout.write("announce route 100.10.0.0/24 next-hop self\n")
sys.stdout.flush()

# Read ACK (new in 4.x, with robust polling)
if not wait_for_ack():
    sys.exit(1)  # Command failed
```

---

### Step 7: Test in Non-Production

```bash
# Start ExaBGP 4.x
exabgp /etc/exabgp/exabgp.conf

# Verify:
# 1. Configuration loads without errors
# 2. BGP session establishes
# 3. API programs run without hanging
# 4. Routes are announced correctly
# 5. Updates are received correctly (if using JSON)
```

**Check logs:**
```bash
tail -f /var/log/exabgp.log
```

---

### Step 8: Deploy to Production

**Deployment strategy:**

1. **Rolling upgrade** (if possible):
   - Upgrade one ExaBGP instance at a time
   - Verify BGP sessions and routes
   - Move to next instance

2. **Blue/Green deployment**:
   - Run 4.x alongside 3.x
   - Gradually shift traffic to 4.x
   - Retire 3.x when stable

3. **Maintenance window**:
   - Schedule downtime
   - Upgrade all instances
   - Verify before restoring service

---

## Testing Your Migration

### Checklist

**Configuration:**
- [ ] Configuration file loads without errors
- [ ] All neighbors defined correctly
- [ ] Process blocks use correct syntax
- [ ] Families match your requirements

**BGP Sessions:**
- [ ] Sessions establish successfully
- [ ] Routes are exchanged
- [ ] Attributes are correct (next-hop, AS-PATH, etc.)
- [ ] FlowSpec rules work (if using)

**API Programs:**
- [ ] Programs start without errors
- [ ] Commands are accepted
- [ ] Programs don't hang
- [ ] ACK responses handled (if enabled)
- [ ] JSON parsing works (if using)

**Monitoring:**
- [ ] Logs show no errors
- [ ] Route counts match expectations
- [ ] No unexpected BGP session resets
- [ ] API programs remain running

---

## Rollback Plan

### If Migration Fails

**Quick rollback:**
```bash
# Stop ExaBGP 4.x
systemctl stop exabgp

# Reinstall 3.x
pip uninstall exabgp
pip install exabgp==3.4.30  # Last 3.x version

# Restore configuration
cp /etc/exabgp/exabgp.conf.3x.backup /etc/exabgp/exabgp.conf

# Restart
systemctl start exabgp
```

**Verify rollback:**
```bash
exabgp version
# Should show: ExaBGP 3.4.x

# Check BGP sessions
exabgp-cli show neighbor summary
```

---

## Common Issues

### Issue #1: Configuration Parse Errors

**Symptom:**
```
ERROR: Could not parse configuration
ERROR: Syntax error at line X
```

**Solution:**
- Check process block uses `api { processes [ ... ]; }` syntax
- Ensure `encoder` directive present in process blocks
- Verify neighbor blocks use correct keywords

---

### Issue #2: API Programs Hang

**Symptom:**
- API program starts but becomes unresponsive
- ExaBGP stops processing commands

**Cause:** ACK feature enabled (default in 4.x), but program doesn't read responses

**Solution:**

**Option 1 - Disable ACK:**
```bash
# Environment variable (works on 4.x and 5.x)
export exabgp.api.ack=false
exabgp /etc/exabgp/exabgp.conf

# OR runtime command (5.x/main only)
# Send: disable-ack or silence-ack
```

**Option 2 - Read ACK responses:**
```python
# Add ACK handling to your program
import select
ready, _, _ = select.select([sys.stdin], [], [], 5.0)
if ready:
    response = sys.stdin.readline().strip()
```

---

### Issue #3: JSON Parsing Fails

**Symptom:**
```
KeyError: 'announce'
KeyError: 'neighbor'
```

**Cause:** JSON structure changed in 4.x

**Solution:** Update JSON parser to use new structure:
```python
# 3.x path
msg['announce']['ipv4 unicast']

# 4.x path
msg['neighbor']['message']['update']['announce']['ipv4 unicast']
```

---

### Issue #4: Routes Not Announced

**Symptom:**
- API program runs
- No errors in logs
- Routes not appearing on router

**Possible causes:**
1. **ACK responses not read** - Program hangs, commands not processed
2. **Family mismatch** - Neighbor not configured for correct address family
3. **Next-hop unreachable** - Router can't reach next-hop IP
4. **BGP session not established** - Check session state

**Debug:**
```bash
# Check ExaBGP logs
tail -f /var/log/exabgp.log

# Check BGP session
exabgp-cli show neighbor 192.168.1.1

# Check process is running
ps aux | grep announce.py
```

---

## Version Comparison Table

| Feature | 3.x | 4.x |
|---------|-----|-----|
| Configuration Syntax | Old format | New format (breaking) |
| JSON Output | Simple structure | Detailed structure (breaking) |
| ACK Responses | ❌ No | ✅ Yes (default) |
| Python 2 | ✅ Supported | ⚠️ Deprecated |
| Python 3 | ⚠️ Limited | ✅ Fully supported |
| FlowSpec | Basic | Full RFC 5575 |
| EVPN | ❌ No | ✅ Yes (RFC 7432) |
| BGP-LS | ❌ No | ✅ Yes (RFC 7752) |
| ADD-PATH | ❌ No | ✅ Yes (RFC 7911) |
| Extended Next-Hop | ❌ No | ✅ Yes (RFC 5549) |
| Performance | Good | Better |
| Maintenance | ❌ Deprecated | ✅ Active |

---

## After Migration

### Verify Everything Works

**Monitor for 24-48 hours:**
- BGP sessions remain stable
- Routes are correct
- No unexpected session resets
- API programs remain running
- No memory leaks
- Logs show no errors

### Update Documentation

- Document your 4.x configuration
- Update runbooks with 4.x commands
- Train team on 4.x differences
- Update monitoring/alerting for 4.x

### Next Steps

Once stable on 4.x:
- **Stay on 4.x for production** (recommended)
- Consider 5.x/main only if you need bleeding-edge features
- 4.x and 5.x are fully compatible (no breaking changes)

---

## Getting Help

### Resources

- **[Version Differences](First-BGP-Session#exabgp-version-differences)** - Complete 3.x vs 4.x comparison
- **[Configuration Syntax](Configuration-Syntax)** - 4.x configuration reference
- **[API Overview](API-Overview)** - 4.x API documentation
- **[Common Pitfalls](Common-Pitfalls)** - Troubleshooting guide

### Community Support

- **GitHub Issues**: https://github.com/Exa-Networks/exabgp/issues
- **Slack**: https://exabgp.slack.com/
- **Mailing List**: See GitHub repo

---

## See Also

- **[Migration Guide](Migration-Guide)** - Main migration entry page
- **[Breaking Changes Reference](Breaking-Changes)** - All breaking changes
- **[4.x to 5.x Migration](From-4.x-to-5.x)** - Upgrading to development branch (optional)

---