graphql

Building a Scalable GraphQL API with Apollo Server and DataLoader

Learn how to build an efficient GraphQL API using Apollo Server and DataLoader. We'll create a blog-like system with users, posts, and comments while avoiding the N+1 query problem.

Code copied!

🎯 What We're Building

We'll create a GraphQL API that handles relationships between:

• Users who write posts and comments
• Posts that belong to users and have comments
• Comments that belong to posts and users

User ──┬── Posts
       └── Comments ── Posts
            

🚀 Getting Started

First, let's set up our project with the necessary dependencies:

package.json

{
  "name": "graphql-apollo-tutorial",
  "version": "1.0.0",
  "type": "module",
  "dependencies": {
    "@apollo/server": "^4.9.5",
    "dataloader": "^2.2.2",
    "graphql": "^16.8.1"
  }
}

📦 Setting Up DataLoaders

DataLoader is a key tool for solving the N+1 query problem in GraphQL. It batches and caches database lookups:

What is the N+1 Problem?
When fetching a list of items and their relations, you might end up making one query for the list and N additional queries for each item's relations.

dataloaders/index.js

import DataLoader from 'dataloader';

export const createLoaders = () => ({
  userLoader: new DataLoader(async (userIds) => {
    console.log('Batch loading users:', userIds);
    return userIds.map(id => users.find(user => user.id === id));
  }),

  userPostsLoader: new DataLoader(async (userIds) => {
    console.log('Batch loading posts for users:', userIds);
    const postsByUser = groupBy(posts, post => post.userId);
    return userIds.map(userId => postsByUser[userId] || []);
  }),

  postCommentsLoader: new DataLoader(async (postIds) => {
    console.log('Batch loading comments for posts:', postIds);
    const commentsByPost = groupBy(comments, comment => comment.postId);
    return postIds.map(postId => commentsByPost[postId] || []);
  })
});

📝 Defining the Schema

Our GraphQL schema defines the types and their relationships:

schema.js

type User {
  id: ID!
  name: String!
  email: String!
  posts: [Post!]!
}

type Post {
  id: ID!
  title: String!
  content: String!
  author: User!
  comments: [Comment!]!
}

type Comment {
  id: ID!
  text: String!
  post: Post!
  author: User!
}

🔧 Implementing Resolvers

Resolvers define how to fetch the data for each field:

resolvers.js

export const resolvers = {
  Query: {
    user: (_, { id }, { loaders }) => {
      return loaders.userLoader.load(id);
    },
    posts: (_, __, { posts }) => posts,
  },

  User: {
    posts: (parent, _, { loaders }) => {
      return loaders.userPostsLoader.load(parent.id);
    },
  },

  Post: {
    author: (parent, _, { loaders }) => {
      return loaders.userLoader.load(parent.userId);
    },
    comments: (parent, _, { loaders }) => {
      return loaders.postCommentsLoader.load(parent.id);
    },
  }
};

🌟 Example Queries

Here's how to query our API:

Query Example

query {
  user(id: "1") {
    name
    email
    posts {
      title
      content
      comments {
        text
        author {
          name
        }
      }
    }
  }
}

🎉 Benefits of This Approach

• Efficient data loading with DataLoader's batching and caching
• Clean separation of concerns between schema and resolvers
• Type-safe API with GraphQL's type system
• Flexible querying capabilities for clients

Important Note:
Remember to create new DataLoader instances for each request to prevent data leaks between different users' requests.

🔍 Performance Monitoring

You can monitor DataLoader's effectiveness by looking at the console logs. You should see multiple IDs being loaded in single batches instead of individual queries.

Pro Tip:
Use the Apollo Studio Explorer (available at http://localhost:4000) to test your queries and see the exact data being requested and returned.

Spring Batch Tutorial: Processing Records with Scheduled Jobs

Learn how to implement a Spring Batch application that processes records every 5 minutes using an in-memory H2 database.

Code copied!

🎯 Project Overview

In this tutorial, we'll build a Spring Batch application that:

• Uses an in-memory H2 database for storing transactions
• Processes pending transactions every 5 minutes
• Implements a complete batch processing pipeline
• Generates sample data for testing

🛠️ Project Setup

First, let's set up our project with the required dependencies in pom.xml:

pom.xml

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-batch</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>
    <dependency>
        <groupId>com.h2database</groupId>
        <artifactId>h2</artifactId>
        <scope>runtime</scope>
    </dependency>
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <optional>true</optional>
    </dependency>
</dependencies>

📦 Domain Model

We'll create a simple Transaction entity to represent our data:

Transaction.java

@Entity
@Data
@NoArgsConstructor
public class Transaction {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String accountId;
    private BigDecimal amount;
    private String type;
    private LocalDateTime timestamp;
    private String status;
}

⚙️ Batch Configuration

The batch job configuration is the heart of our application:

BatchConfig.java

@Configuration
@EnableBatchProcessing
@RequiredArgsConstructor
public class BatchConfig {
    @Bean
    public ItemReader reader() {
        RepositoryItemReader reader = new RepositoryItemReader<>();
        reader.setRepository(transactionRepository);
        reader.setMethodName("findByStatus");
        reader.setArguments(java.util.Arrays.asList("PENDING"));
        return reader;
    }

    @Bean
    public ItemProcessor processor() {
        return transaction -> {
            transaction.setStatus("PROCESSED");
            return transaction;
        };
    }

    @Bean
    public Job processTransactionsJob() {
        return jobBuilderFactory.get("processTransactionsJob")
                .incrementer(new RunIdIncrementer())
                .flow(processTransactionsStep())
                .end()
                .build();
    }
}

⏰ Job Scheduler

The scheduler runs our batch job every 5 minutes:

BatchJobScheduler.java

@Component
@EnableScheduling
@RequiredArgsConstructor
@Slf4j
public class BatchJobScheduler {
    @Scheduled(fixedRate = 300000) // Run every 5 minutes
    public void runBatchJob() {
        JobParameters jobParameters = new JobParametersBuilder()
                .addLong("time", System.currentTimeMillis())
                .toJobParameters();
        
        jobLauncher.run(processTransactionsJob, jobParameters);
    }
}

🔄 Sample Data Generation

We generate sample transactions on application startup:

TransactionService.java

@Service
@RequiredArgsConstructor
public class TransactionService {
    public void generateSampleTransactions() {
        for (int i = 0; i < 50; i++) {
            Transaction transaction = new Transaction();
            transaction.setAccountId(UUID.randomUUID().toString());
            transaction.setAmount(BigDecimal.valueOf(Math.random() * 1000));
            transaction.setType(Math.random() > 0.5 ? "CREDIT" : "DEBIT");
            transaction.setTimestamp(LocalDateTime.now());
            transaction.setStatus("PENDING");
            
            transactionRepository.save(transaction);
        }
    }
}

💡 Key Features

This implementation includes:

• Chunk-based processing (10 items per chunk)
• Automatic job parameter incrementation
• Error handling and logging
• Scheduled execution every 5 minutes
• In-memory H2 database for easy testing

🚀 Running the Application

1. Clone the repository
2. Run mvn clean install
3. Start the application
4. Access H2 console at http://localhost:8080/h2-console
5. Watch the logs for batch job execution every 5 minutes

Note: The H2 database is in-memory, so data will be reset when the application restarts.

🔍 Monitoring

You can monitor the batch jobs through:

• Application logs
• H2 console for database inspection
• Spring Batch tables (BATCH_JOB_EXECUTION, BATCH_STEP_EXECUTION)