Real-time Streaming Analysis with IMDv3
This tutorial demonstrates how to use MDAnalysis for real-time streaming analysis of molecular dynamics simulations using the Interactive Molecular Dynamics (IMD) v3 protocol. You’ll learn how to connect to running simulations and perform live analysis as the simulation progresses.
Streaming involves processing data in real-time as it is generated, rather than storing it for later analysis. In molecular dynamics, this means sending simulation data to a client on-the-fly while the simulation is running, without writing large trajectory files to disk.
This is achieved through a TCP/IP socket connection between the simulation engine and receiving client, transmitting coordinates, velocities, forces, energies, and timing information using the IMDv3 protocol.
What it covers
How to set up streaming connections to MD engines
Real-time monitoring
Live analysis workflows
Prerequisites
Before starting, you’ll need:
MDAnalysis with IMD support
The
imdclientpackage (≥ 0.2.2)A running MD simulation with IMD enabled (examples are engine agnostic for the most part)
Installation and Setup
The IMDReader requires the imdclient package. Let’s check if everything is properly installed:
[1]:
# Install required packages (uncomment if needed)
# !pip install imdclient>=0.2.2
import warnings
warnings.filterwarnings('ignore')
import MDAnalysis as mda
import numpy as np
import matplotlib.pyplot as plt
from datetime import datetime
import time
# Check if IMD support is available
try:
from MDAnalysis.coordinates.IMD import IMDReader, HAS_IMDCLIENT
print(f"IMD support available: {HAS_IMDCLIENT}")
if HAS_IMDCLIENT:
import imdclient
print(f"imdclient version: {imdclient.__version__}")
print("✅ Ready for streaming analysis!")
else:
print("❌ IMD support not available")
except ImportError as e:
print(f"❌ IMD support not available: {e}")
print("Please install imdclient: pip install imdclient>=0.2.2")
IMD support available: False
❌ IMD support not available
Setting Up a Simulation with IMD
Before we can demonstrate streaming analysis, we need a simulation running with IMD enabled. Here are configuration examples for different MD engines:
GROMACS Setup
Add these comprehensive IMD settings to your .mdp file:
; IMD settings for v3 protocol
IMD-group = System ; Group to stream (typically System)
IMD-version = 3 ; Use IMDv3 protocol (required for MDAnalysis)
IMD-nst = 1 ; Frequency of data transmission (every step)
IMD-time = No ; Send time information
IMD-coords = Yes ; Send atomic coordinates (essential)
IMD-vels = No ; Send velocities (optional)
IMD-forces = No ; Send forces (optional)
IMD-box = No ; Send box dimensions (optional)
IMD-unwrap = No ; Unwrap coordinates across PBC
IMD-energies = No ; Send energy information (optional)
Run the simulation:
gmx mdrun -v -nt 4 -imdwait -imdport 8889
LAMMPS Setup
Use the comprehensive IMD fix in your input script:
# IMD setup - full parameter specification
fix ID group-ID imd <port> trate <frequency> version 3 unwrap <on/off> time <on/off> box <on/off> coordinates <on/off> velocities <on/off> forces <on/off>
# Example with specific values:
fix imd all imd 8889 trate 1 version 3 unwrap on time on box on coordinates on velocities on forces on
Run your LAMMPS simulation as usual.
NAMD Setup
Add comprehensive IMD configuration to your NAMD configuration file:
# IMD Settings
IMDon yes
IMDport 8889 # Must match client port
IMDwait yes # Wait for client connection
IMDfreq 1 # Frequency of sending data
# Data transmission settings
IMDsendPositions yes # Send atomic coordinates
IMDsendEnergies yes # Send energy information
IMDsendTime yes # Send timing data
IMDsendBoxDimensions yes # Send simulation box info
IMDsendVelocities yes # Send atomic velocities
IMDsendForces yes # Send atomic forces
IMDwrapPositions no # Don't wrap coordinates
Run your NAMD simulation as usual.
Example 1: Simple IMD Analysis
This example shows the basics of connecting to a live simulation and performing simple analysis.
[2]:
# Simple Example: Basic IMD Connection and Analysis
import MDAnalysis as mda
import numpy as np
def simple_imd_analysis():
"""
Simple example of connecting to a live simulation and analyzing it.
This assumes you have GROMACS running with IMD enabled on localhost:8889
"""
try:
# Connect to the live simulation
u = mda.Universe("topol.tpr", "imd://localhost:8889")
print(f"Connected to simulation: {u.atoms.n_atoms} atoms")
# Basic analysis on streaming frames
protein = u.select_atoms("protein")
frame_count = 0
for ts in u.trajectory:
# Calculate radius of gyration
rg = protein.radius_of_gyration()
print(f"Frame {frame_count}: Time = {ts.time:.1f} ps, Rg = {rg:.2f} Å")
frame_count += 1
# Stop after 10 frames for this simple example
if frame_count >= 10:
break
print("Simple analysis completed!")
except Exception as e:
print(f"Connection failed: {e}")
print("Make sure your simulation is running with IMD enabled")
print("Example: gmx mdrun -s topol.tpr -imdport 8889 -imdwait")
# To run this example, uncomment the line below:
# simple_imd_analysis()
Example 2: Advanced Live Streaming with Real-time Plotting
This example demonstrates continuous monitoring with live plots that update as the simulation runs.
[3]:
# Advanced Example: Live Streaming Analysis with Real-time Plotting
import MDAnalysis as mda
import matplotlib.pyplot as plt
import numpy as np
from collections import deque
import time
def advanced_live_streaming():
"""
Advanced example with live plotting and continuous monitoring.
This connects to your simulation and creates real-time plots.
"""
try:
# Connect to the simulation
u = mda.Universe("system.tpr", "imd://localhost:8889")
print(f"Starting live analysis of {u.atoms.n_atoms} atoms")
# Setup for real-time plotting
plt.ion() # Interactive mode
fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(10, 8))
# Data storage for plotting
times = deque(maxlen=100) # Keep last 100 points
rg_values = deque(maxlen=100)
energies = deque(maxlen=100)
protein = u.select_atoms("protein")
frame_count = 0
for ts in u.trajectory:
current_time = ts.time
# Calculate properties
rg = protein.radius_of_gyration()
# Simulate potential energy (real simulations would have this)
# In real IMD, you might get this from the simulation engine
potential_energy = -1000 + np.random.normal(0, 50)
# Store data
times.append(current_time)
rg_values.append(rg)
energies.append(potential_energy)
# Update plots every 5 frames
if frame_count % 5 == 0:
# Clear and replot
ax1.clear()
ax2.clear()
# Plot radius of gyration
ax1.plot(list(times), list(rg_values), 'b-', linewidth=2)
ax1.set_xlabel('Time (ps)')
ax1.set_ylabel('Radius of Gyration (Å)')
ax1.set_title('Real-time Rg Evolution')
ax1.grid(True)
# Plot energy
ax2.plot(list(times), list(energies), 'r-', linewidth=2)
ax2.set_xlabel('Time (ps)')
ax2.set_ylabel('Potential Energy (kJ/mol)')
ax2.set_title('Real-time Energy Evolution')
ax2.grid(True)
plt.tight_layout()
plt.draw()
plt.pause(0.01) # Small pause to update display
# Print status
print(f"Frame {frame_count}: Time = {current_time:.1f} ps, "
f"Rg = {rg:.2f} Å, Energy = {potential_energy:.1f} kJ/mol")
frame_count += 1
# Run for 50 frames in this example
if frame_count >= 50:
break
print("Live streaming analysis completed!")
plt.ioff() # Turn off interactive mode
plt.show()
except KeyboardInterrupt:
print("\nAnalysis stopped by user")
plt.ioff()
except Exception as e:
print(f"Error during live analysis: {e}")
print("Make sure your simulation is running with IMD enabled")
plt.ioff()
# To run this advanced example, uncomment the line below:
# advanced_live_streaming()