This is my personal Blog

Ivan Zoratti

  • Yes, Neo4j Scales: Welcoming Infinigraph to the Graph Intelligence Platform

    Let’s be honest about graph databases. For a long time, there was a running joke (or at least a collective anxiety) among DBAs and DevOps engineers: “Graphs are amazing for small, highly connected datasets, but the moment you hit massive scale, good luck.”

    We’ve all heard the warnings. You build a beautiful graph model on your laptop, everything runs flawlessly in Docker, and then the data science or fraud team tells you they need to scale it to tens or hundreds of terabytes. Suddenly, you’re sweating over RAM limitations, vertical scaling costs, and the absolute nightmare of trying to shard a graph without breaking your traversals.

    Well, you can officially put that anxiety to rest. Neo4j has introduced Infinigraph, a brand-new distributed architecture integrated into the Graph Intelligence Platform.

    The headline? Neo4j does scale. Seamlessly. Horizontally. Into the hundreds of terabytes.

    Here is exactly what this means for your architecture, your deployment pipelines, and your sanity.

    The Elephant in the Server Room: Why Graph Scaling Used to Hurt

    Traditional database sharding is relatively straightforward: you split your users by ID or your logs by date, and you push them onto separate machines.

    But a graph is defined by its connections. If you blindly slice a graph across multiple servers, a single multi-hop query (like looking for a fraud ring or tracing a supply chain) forces your database to constantly hop across network boundaries. Your query performance plummets, your network saturates, and your DevOps team starts updating their resumes.

    Because of this, engineering teams often resorted to complex workarounds: separating operational (OLTP) and analytical (OLAP) workloads, building massive ETL pipelines, and duplicating storage.

    Enter Infinigraph: Real Distributed Architecture

    Infinigraph changes the game by introducing a true distributed graph architecture that scales to 100TB+ workloads. It handles the scaling bottleneck through a clever engineering approach called property sharding. Instead of ripping the core topology of your graph apart, Infinigraph distributes the massive property sets, wide metadata documents, and heavy vector embeddings across a cluster while keeping the core graph structure logically whole.

    Here is how it actually works under the hood inside a Neo4j Autonomous Cluster:

    • The Graph Shard (The Core Topology): One specialized shard stores the absolute essentials—nodes, relationships, labels, and unique identifiers. Because it isn’t bogged down by heavy text or vectors, this shard stays lean and fits into memory for lightning-fast traversals.
    • The Property Shards (The Massive Distribution): Properties are separated, hashed, and distributed evenly across a cluster of specialized property shards.
    • Full High Availability & Fault Tolerance: The graph shard forms a core Raft group that manages transactions, availability, and failover. Data is then propagated to the property shards via transaction logs. Need more read throughput or redundancy? You can use more shard or scale the graph shards by adding secondaries and the property shards by adding replicas.

    From Laptop to Multi-Node Cluster (Without Changing Code)

    The absolute best part of this architecture for developers and DevOps is deployment predictability. The database commands map directly from your local dev sandbox to enterprise production clusters.

    On your laptop, you can play around with standard single databases. If you want to simulate a sharded layout locally for testing, you can use Neo4j’s composite database tooling. But when you move to production, you unleash the full power of Infinigraph using native CREATE DATABASE cluster syntax.

    Let’s look at how the deployment configuration evolves from a traditional cluster setup to a property-sharded Infinigraph setup:

    Traditional Cluster Database (Non-Sharded)

    // Creates a standard database mirrored across 3 core cluster nodes
CREATE DATABASE foo 
TOPOLOGY 3 PRIMARIES;

    Infinigraph Distributed Database (Property-Sharded)

    // Creates a graph topology shard backed by 4 property shards,
// each configured with 2 replicas for high availability
CREATE DATABASE foo
   SET GRAPH SHARD { TOPOLOGY 3 PRIMARIES }
   SET PROPERTY SHARDS { COUNT 4 TOPOLOGY 2 REPLICAS };

    From the application’s perspective, nothing changes. Your drivers connect to the cluster exactly the same way. Your developers write the exact same Cypher queries. The graph engine automatically handles routing data to the correct shards behind the scenes.

    Real-World Use Cases: Where This Saves the Day

    To see why this is a game-changer, let’s look at two massive architectural patterns that used to break traditional graph setups.

    1. GenAI & GraphRAG at Scale (The Social Network / Identity Layer)

    Imagine building a global network application: think massive social graphs or enterprise identity layers mimicking an LDBC/SNB benchmark dataset. You have hundreds of millions of users, deep behavioral connections, and a massive footprint of full-text metadata. To make things more intense, you add GenAI capabilities, hanging heavy vector embeddings (like a 1536-dimensional array) off every single node for semantic graph searches.

    • Without Infinigraph: the vector indexes and property bloat choke your page cache. Your cluster nodes stall because they are trying to load gigabytes of vector data into RAM alongside the graph layout.
    • With Infinigraph: your graph shard processes structural queries and multi-hop traversals instantly. The massive vector arrays, text payloads, and bio documents live comfortably across your property shards.

    2. High-Throughput Real-Time Fraud Engines

    Picture a financial transaction graph where thousands of credit card transactions stream into the system every second. Every transaction node carries deep JSON metadata payloads, merchant locations, risk scores, and device fingerprints.

    • Without Infinigraph: High-concurrency writes stall because the single node is busy writing massive blobs of transaction metadata to disk while simultaneously trying to calculate fraud rings in real time.
    • With Infinigraph: Full ACID compliance ensures data integrity across the cluster. Transactions write smoothly because the heavy metadata payloads are hashed and spread across the property shards via transaction logs. Your fraud analysts run real-time queries against the lean graph topology shard without hitting a performance ceiling.

    The Bottom Line

    If you’ve been holding back on using graph technology for your heaviest enterprise datasets because you were worried about hitches at scale, the ceiling has officially been removed. With Infinigraph now available for property-heavy workloads, you can confidently design large-scale Knowledge Graphs, real-time fraud engines, or massive GraphRAG applications knowing the database will grow with your data.

    The era of the infinite graph is here. Go build something massive.

  • Seven Years… Yes, Seven Years!

    Time flies. I feel like only few months ago I was sitting at Caffè Trieste, my favourite spot in San Francisco, or at Clairmont’s in Windsor, where I lived for more than 21 years. Those two places inspired me most, and from them I wrote the majority of my blogs and articles, and designed my products.

    What happened since then? In 2018 I was frequently traveling to San Francisco, where I co-founded Dianomic Systems Inc., a company focused on collecting and managing data for IIoT. I was flying back and forth, often staying longer than just a few days in order to adapt to the time zone and function “properly.” A word of advice: don’t ever do that. It is poison for your body—a constant reset of your biological clock. On my way back to the UK one time, I fainted from exhaustion, headaches, and muscle pain. When I finally made it home, I slept for 48 hours straight. That was my wake-up call: I had to make a big decision. Either stay with my family in Windsor and close to my parents in Milan, or spend most of my time in the Bay Area. I chose the former. With the sadness that comes from having to leave behind something deeply important, I stepped away from Dianomic and joined Neo4j, which allowed me to work from London and travel “less.”

    Could I have traveled less and worked more from home while still supporting a startup? Perhaps—but if you’ve ever built one, you know the stress and sheer energy it requires. I didn’t feel comfortable giving less than what I thought was necessary to nurture a newborn company. On the other hand, Neo4j was a strong attraction. It meant going back to my roots in the database world. I liked the people, I liked the product, and I could see there was a lot of important work to do.

    Here I am, seven years later. Neo4j has nearly quintupled in size. I now have a team helping me manage the product. Engineering has become a wonderful, powerful machine with established processes. In these seven years, just my team alone has delivered a wide range of features:

    Now, seven years on, as I celebrate my 60th birthday and reflect on more than 40 years in the database world (I began by learning and teaching DEC RDB—later Oracle—and building applications with dBase II), I still find joy in innovating and creating new features for this market.

    What’s next?
    You’ll see in upcoming blog posts. I’ve taken on new challenges, perfectly aligned with the spirit of the moment.

    No, I am not jumping on the AI bandwagon by funding a .ai startup or by building AI/GenAI-based services. I believe I can serve the world better by focusing on what I do best: improving and integrating existing technologies to meet both current and future needs. AI will not be truly useful without a solid infrastructure that goes beyond inference servers and search engines. For that, we need to consider two fundamental concepts:

    1. A set of structured facts, best served through knowledge graphs.
    2. A set of rules that can model and achieve goals—whether tasks involve transactions, data generation, or natural-language answers—best delivered through graph intelligence.

    So, stay tuned. Another seven years of innovation may come.