GraphQL
GraphQL is a query language for APIs that lets clients ask for exact data they need, avoiding unnecessary information. Unlike traditional REST APIs (which require multiple endpoints for different resources), GraphQL uses a single endpoint with flexible queries. However, because each field in a query is handled by separate functions (resolvers), it can lead to the N+1 problem: fetching related data (like posts and their authors) might trigger separate database calls for each item, causing inefficiency. To fix this, developers use strategies and tools to optimize data fetching, even for complex nested queries.
Dataloaders
dataloaders solve performance issues in GraphQL, especially the N+1 problem. They work by batching (grouping) and caching requests. For example: if multiple resolvers ask for user data (like post authors), the dataloader collects all required IDs in one processing cycle and makes a single database query instead of one per ID. It also caches results to avoid repeats. This reduces the number of requests and speeds up the API, making dataloaders essential for scalable GraphQL apps.
NestJS
NestJS is a Node.js framework for building organized, scalable apps. It has built-in GraphQL support, letting developers define schemas, resolvers, and use dependency injection for clean code structure. To optimize queries, developers often use dataloaders in resolvers. However, a common challenge is needing to create a separate dataloader for every data relationship (e.g., one for users, another for posts). This leads to repetitive code and complexity in large projects. Despite this, NestJS's modular architecture and powerful tools make it a top choice for building complex, high-performance APIs.
NestJS Decorated Dataloaders
A lightweight wrapper around Dataloader that lets you declare where to batch and cache instead of wiring it by hand. Add a @Load decorator to any field, register a handler, and the N+1 query problem is gone.
Key Features
Installation
npm install nestjs-decorated-dataloaders
Quick Start
Module Configuration
Configure the DataloaderModule
in your application module:
import { Module } from "@nestjs/common";
import { GraphQLModule } from "@nestjs/graphql";
import { LRUMap } from "lru_map";
import { DataloaderModule } from "nestjs-decorated-dataloaders";
@Module({
imports: [
GraphQLModule.forRoot({
autoSchemaFile: true,
}),
DataloaderModule.forRoot({
cache: true,
maxBatchSize: 100,
getCacheMap: () => new LRUMap(100),
name: "MyAwesomeDataloader",
}),
],
})
export class AppModule {}
- cache: Enables caching.
- maxBatchSize: Limits the maximum number of batched requests.
- getCacheMap: Defines a custom cache implementation (e.g., LRU Cache).
- name: Names the dataloader for better tracking and debugging.
Defining Entities
PhotoEntity
export class PhotoEntity {
id: number;
url: string;
userId: number;
}
UserEntity
import { Load } from "nestjs-decorated-dataloaders";
import { PhotoEntity } from "./photo.entity";
export class UserEntity {
id: number;
name: string;
@Load(() => PhotoEntity, { key: "id", parentKey: "userId", handler: "LOAD_PHOTOS_BY_USER_ID" })
photo: PhotoEntity;
@Load(() => [PhotoEntity], { key: "id", parentKey: "userId", handler: "LOAD_PHOTOS_BY_USER_ID" })
photos: PhotoEntity[];
}
Dataloader Handlers
Dataloader handlers define how data is fetched from the data source. Handlers are tied to specific dataloaders using the @DataloaderHandler
decorator.
import { DataloaderHandler } from "nestjs-decorated-dataloaders";
import { PhotoEntity } from "./photo.entity";
export class PhotoService {
@DataloaderHandler("LOAD_PHOTOS_BY_USER_ID")
async loadPhotosByUserIds(userIds: number[]): Promise<PhotoEntity[]> {
// Replace with actual data fetching logic
}
}
Using Dataloaders in Resolvers
Resolvers use the DataloaderService
to load related entities, ensuring requests are batched and cached.
import { Resolver, ResolveField, Parent } from "@nestjs/graphql";
import { DataloaderService } from "nestjs-decorated-dataloaders";
import { UserEntity } from "./user.entity";
import { PhotoEntity } from "./photo.entity";
@Resolver(UserEntity)
export class UserResolver {
constructor(private readonly dataloaderService: DataloaderService) {}
@ResolveField(() => PhotoEntity)
async photo(@Parent() user: UserEntity) {
return this.dataloaderService.load({ from: UserEntity, field: "photo", data: user });
}
@ResolveField(() => [PhotoEntity])
async photos(@Parent() user: UserEntity) {
return this.dataloaderService.load({ from: UserEntity, field: "photos", data: user });
}
}
Advanced Concepts
Function-Based Mapper
Function-Based Mapper allows you to use functions instead of string paths for the key
and parentKey
properties in the @Load
decorator. This is particularly useful when you need to work with composite keys or when you need more complex mapping logic.
import { Field, Int, ObjectType } from "@nestjs/graphql";
import { Load } from "nestjs-decorated-dataloaders";
import { CategoryPostEntity } from "./category-post.entity";
import { CategoryEntity } from "./category.entity";
@ObjectType()
export class PostEntity {
@Field(() => Int)
id: number;
@Field(() => String)
title: string;
@Field(() => String)
content: string;
@Field(() => String)
createdAt: string;
// Relationship with CategoryPostEntity for the many-to-many relationship
categoryPosts: CategoryPostEntity[];
/**
* Using Function-Based Mapper for complex relationships
* This handles a many-to-many relationship through a join table
*/
@Load(() => [CategoryEntity], {
key: (category) => category.id,
parentKey: (post) => post.categoryPosts.map((cp) => cp.postId),
handler: "LOAD_CATEGORY_BY_POSTS",
})
categories: CategoryEntity[];
}
Benefits of Function-Based Mapper
- Complex Mapping: You can implement complex mapping logic that goes beyond simple property access.
- Composite Keys: You can create composite keys by combining multiple fields.
- Flexibility: You can use any JavaScript expression to compute the key.
- Performance: Function-based mappers are more CPU efficient compared to string-based mappers.
Type Safety
You can use TypeScript generics to ensure type safety when declaring a Dataloader field.
import { Field, Int, ObjectType } from "@nestjs/graphql";
import { Load } from "nestjs-decorated-dataloaders";
import { PhotoEntity } from "./photo.entity";
@ObjectType()
export class UserEntity {
@Field(() => Int)
id: number;
@Field(() => String)
name: string;
@Field(() => Date)
createdAt: Date;
@Load<PhotoEntity, UserEntity>(() => [PhotoEntity], {
key: (user) => user.id,
parentKey: (photo) => photo.userId,
handler: "LOAD_PHOTOS_BY_USER",
})
photos: Array<PhotoEntity>;
}
In this example, the key
function is typed to receive a UserEntity
and the parentKey
function is typed to receive a PhotoEntity
.
Handling Circular Dependencies
Circular dependencies between entities (e.g., User ↔ Photo) can cause metadata resolution errors when using reflect-metadata. For example:
- reflect-metadata tries to read metadata from User, which references Photo.
- Photo in turn references User, but if User hasn't been fully initialized, its metadata resolves to undefined.
This issue is common in environments using SWC. To resolve it, use the Relation<T>
wrapper provided by nestjs-decorated-dataloaders.
Solution: Wrapping Circular References
Encapsulate circular properties with Relation<T>
. This prevents reflect-metadata from attempting to resolve the circular dependency during type introspection.
import { Relation } from 'nestjs-decorated-dataloaders';
class User {
photo: Relation<Photo>;
}
class Photo {
user: Relation<User>;
}
How It Works
- Generic Type Erasure: reflect-metadata cannot infer generic types like
Relation<Photo>
, so it defaults the metadata to undefined, avoiding circular resolution errors. - Explicit Type Declaration: You must manually specify the wrapped type (e.g.,
Relation<Photo>
) to retain type safety in your code.
Important Notes
- Use
Relation<T>
only for circular dependencies. For non-circular references, use direct types (e.g.,Photo
instead ofRelation<Photo>
). - Ensure the generic type (e.g.,
Photo
insideRelation<Photo>
) is explicitly declared to avoid type inference issues.
Aliases
Aliases allow you to link a dataloader handler to an abstract class, which is especially useful when working with more complex architectures that include abstract or shared classes.
Why Use Aliases?
Sometimes you may want to map a dataloader handler to an abstract class that doesn't allow decorators. Aliases provide a way to assign a handler to such cases.
Using Aliases
@AliasFor(() => AbstractPhotoService)
export class ConcretePhotoService {}
This allows PhotoService
to serve as the dataloader handler for AbstractPhotoService
.
Under the Hood
nestjs-decorated-dataloaders
is built on top of the GraphQL Dataloader library. At its core, a dataloader is a mechanism for batching and caching database or API requests, reducing the number of round trips required to fetch related data.
- Batching: Dataloader batches multiple requests for the same resource into a single query. This ensures that, rather than issuing one query per entity (e.g., fetching one photo per user), the dataloader combines them into a single query that fetches all the photos for the users in one go.
- Caching: Dataloader caches query results, preventing redundant queries for the same data within the same request cycle. This ensures that once a resource is fetched, subsequent requests for the same resource will use the cached data.
High-Level Nest.js Abstraction
nestjs-decorated-dataloaders
abstracts the complexities of manually managing dataloaders and integrates seamlessly with Nest.js using decorators. It provides a declarative and maintainable approach to solving the N+1 problem, allowing you to focus on building features without worrying about the underlying dataloader logic.
By using decorators like @Load
and @DataloaderHandler
, this module streamlines dataloader setup, making it simple to handle related entities in GraphQL resolvers without manual dataloader instantiation or dependency injection.