Managing Your Data: A Practical Guide to TypeORM in a NestJS Microservice Architecture

Hey there, microservice maestros! We've come a long way. From understanding the "why" of microservices, to setting up our monorepo, to mastering synchronous gRPC communication, and then diving into the awesome world of asynchronous event-driven patterns with RabbitMQ. Our services are talking, they're decoupled, and they're resilient!

But there's a giant elephant in the room when it comes to backend applications: data. Where does all this precious information live? How do our independent microservices interact with it? This is often one of the trickiest parts of moving from a monolith to microservices.

In a monolith, it's simple: one big database, all services talk to it. Easy peasy. But in a microservice world, that approach can quickly become a tangled mess, undermining all the benefits we've worked so hard to achieve. If multiple services directly access the same tables, they become tightly coupled to each other's database schemas. A change in one service's data model could break another service, completely defeating the purpose of independent deployment.

The Golden Rule: Database Per Service

The ideal, purest form of microservice data management is the "database per service" pattern. This means each microservice owns its own private database. No other service is allowed to directly access that database. If another service needs data from it, it must go through the owning service's API (e.g., via gRPC or events).

This provides ultimate autonomy:

• Independent Schema Evolution: A service can change its internal database schema without affecting any other service.
• Technology Freedom: Each service can choose the best database technology for its specific needs (e.g., PostgreSQL for relational data, MongoDB for document data, Redis for caching, Neo4j for graph data).
• Strong Data Ownership: Clear boundaries for who is responsible for what data.

However, in the real world, especially when migrating from a monolith or starting with a smaller microservice footprint, you might encounter scenarios where multiple services appear to use the same database instance. The key here is logical separation and strict ownership. Even if they share the same physical database server, each service should only interact with its own set of tables, and other services should never bypass the owning service's API to access that data.

For our users-service, we'll set up a PostgreSQL database. While other services won't directly touch it, the users-service will be the sole owner of the user data within that database.

Setting Up PostgreSQL with Docker

As always, Docker is our best friend for quickly spinning up dependencies. PostgreSQL is a powerful, open-source relational database system, and it's a fantastic choice for many microservices.

Open your terminal and run the following command from anywhere (it doesn't have to be in your monorepo):

docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -p 5432:5432 -d postgres

What this command does:

• docker run -d: Runs the container in detached mode (in the background).
• --name some-postgres: Gives your container a memorable name (some-postgres).
• -e POSTGRES_PASSWORD=mysecretpassword: Sets the password for the default postgres user. Remember this password!
• -p 5432:5432: Maps port 5432 (the default PostgreSQL port) from the container to your host machine.
• postgres: Specifies the official PostgreSQL Docker image.

Give it a moment to pull the image and start up. You can verify it's running with docker ps.

Integrating TypeORM into the users-service

Now that our database is ready, let's integrate TypeORM into our users-service. TypeORM is an Object Relational Mapper (ORM) that runs in Node.js, TypeScript, JavaScript (ES5, ES6, ES7, ES8), Python, React, Swift, Java, etc. It supports PostgreSQL, MySQL, MariaDB, SQLite, MS SQL Server, Oracle, SAP Hana, Aurora MySQL, Aurora PostgreSQL, CockroachDB, and MongoDB. It allows you to work with your database using TypeScript classes (entities) instead of raw SQL queries, making your code more object-oriented and type-safe.

Step 1: Install Necessary Packages

Navigate into your users-service directory and install TypeORM, the PostgreSQL driver, and a few other helpers:

cd apps/users-service
npm install @nestjs/typeorm typeorm pg @types/pg
# or yarn add @nestjs/typeorm typeorm pg @types/pg
cd ../../ # Go back to the monorepo root

• @nestjs/typeorm: The official NestJS integration for TypeORM.
• typeorm: The core TypeORM library.
• pg: The PostgreSQL driver for Node.js.
• @types/pg: TypeScript type definitions for the PostgreSQL driver.

Step 2: Configure TypeORM in users-service/src/app/app.module.ts

We'll configure the database connection within our AppModule.

// apps/users-service/src/app/app.module.ts
import { Module } from '@nestjs/common';
import { ClientsModule, Transport } from '@nestjs/microservices';
import { join } from 'path';
import { TypeOrmModule } from '@nestjs/typeorm'; // Import TypeOrmModule
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { User } from './user/entities/user.entity'; // Import our User entity (we'll create this next)

@Module({
  imports: [
    // TypeORM configuration for PostgreSQL
    TypeOrmModule.forRoot({
      type: 'postgres',
      host: 'localhost', // Or the IP of your Docker host if running Docker remotely
      port: 5432,
      username: 'postgres', // Default PostgreSQL user
      password: 'mysecretpassword', // Your Docker password
      database: 'postgres', // Default database name
      entities: [User], // Register our User entity
      synchronize: true, // IMPORTANT: Set to false in production! This automatically
                        // creates database tables based on your entities. Great for dev,
                        // dangerous for production data.
    }),
    TypeOrmModule.forFeature([User]), // Register entities for this module
    ClientsModule.register([
      {
        name: 'RABBITMQ_SERVICE',
        transport: Transport.RMQ,
        options: {
          urls: ['amqp://guest:guest@localhost:5672'],
          queue: 'user_events_queue',
          queueOptions: {
            durable: false
          },
        },
      },
    ]),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Explanation:

• TypeOrmModule.forRoot(): This is used to configure the main database connection for your application.
  • type: 'postgres': Specifies the database type.
  • host, port, username, password, database: Your connection details. Remember, localhost here refers to the host machine where Docker is running, as the container's port 5432 is mapped to your host's 5432.
  • entities: [User]: This is where you tell TypeORM which entity classes represent your database tables.
  • synchronize: true: Crucial for development, dangerous for production! When true, TypeORM will automatically create your database schema (tables, columns) based on your entity definitions every time the application starts. This is super convenient for rapid prototyping but can lead to data loss or unexpected schema changes in a production environment. In production, you'd use TypeORM migrations to manage schema changes explicitly.
• TypeOrmModule.forFeature([User]): This registers the User entity for use within this specific module. It makes the UserRepository (which we'll use next) available for injection.

Step 3: Create the User Entity

An entity is a TypeScript class that maps to a database table. Create a new directory apps/users-service/src/user and inside it, create entities/user.entity.ts:

// apps/users-service/src/user/entities/user.entity.ts
import { Entity, PrimaryGeneratedColumn, Column } from 'typeorm';

@Entity() // Marks this class as a TypeORM entity, mapping to a 'user' table by default
export class User {
  @PrimaryGeneratedColumn() // Marks 'id' as the primary key, auto-incrementing
  id: number;

  @Column({ unique: true }) // Marks 'email' as a column, ensuring unique values
  email: string;

  @Column() // Marks 'name' as a column
  name: string;
}

Explanation:

• @Entity(): Decorator that tells TypeORM this class is a database entity. By default, it will map to a table named user (lowercase plural of the class name).
• @PrimaryGeneratedColumn(): Creates an auto-incrementing primary key column.
• @Column(): Marks a property as a database column.
• { unique: true }: An option for @Column to enforce uniqueness at the database level.

Step 4: Update users-service/src/app/app.service.ts to Use TypeORM

Now, let's inject the UserRepository into our AppService and implement actual CRUD operations.

// apps/users-service/src/app/app.service.ts
import { Injectable } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm'; // Import InjectRepository
import { Repository } from 'typeorm'; // Import Repository
import { User } from '../user/entities/user.entity'; // Import User entity

@Injectable()
export class AppService {
  constructor(
    // Inject the TypeORM Repository for the User entity
    @InjectRepository(User)
    private usersRepository: Repository<User>,
  ) {}

  // --- CRUD Operations ---

  async findAllUsers(): Promise<User[]> {
    return this.usersRepository.find(); // Find all users
  }

  async findUserById(id: number): Promise<User | undefined> {
    return this.usersRepository.findOne({ where: { id } }); // Find user by ID
  }

  async createUser(name: string, email: string): Promise<User> {
    const newUser = this.usersRepository.create({ name, email }); // Create a new User instance
    return this.usersRepository.save(newUser); // Save it to the database
  }

  async updateUser(id: number, name?: string, email?: string): Promise<User | undefined> {
    const user = await this.usersRepository.findOne({ where: { id } });
    if (!user) {
      return undefined;
    }
    if (name !== undefined) {
      user.name = name;
    }
    if (email !== undefined) {
      user.email = email;
    }
    return this.usersRepository.save(user); // Save updated user
  }

  async deleteUser(id: number): Promise<boolean> {
    const result = await this.usersRepository.delete(id); // Delete user by ID
    return result.affected > 0; // Return true if a user was deleted
  }
}

Explanation:

• @InjectRepository(User): This decorator injects a TypeORM Repository instance for the User entity. The Repository provides methods for interacting with the database (e.g., find, findOne, create, save, delete).
• findAllUsers(), findUserById(), createUser(), updateUser(), deleteUser(): These are our new CRUD methods, using the usersRepository to perform database operations. Notice how TypeORM abstracts away the SQL, allowing us to work with JavaScript/TypeScript objects.

Step 5: Update users-service/src/app/app.controller.ts to Expose CRUD via gRPC

Now, we'll modify our controller to use the new service methods and expose them via gRPC. We'll need to update our users.proto file first to define the new gRPC methods and their message types.

First, update libs/proto/users.proto:

// libs/proto/users.proto
syntax = "proto3";

package users;

message User {
  int32 id = 1;
  string name = 2;
  string email = 3;
}

message Empty {}

message CreateUserRequest {
  string name = 1;
  string email = 2;
}

// New message type for GetUserById request
message GetUserByIdRequest {
  int32 id = 1;
}

// New message type for UpdateUser request
message UpdateUserRequest {
  int32 id = 1;
  string name = 2; // Optional field, use wrapper types for true optionality
  string email = 3; // Optional field
}

// New message type for DeleteUser request
message DeleteUserRequest {
  int32 id = 1;
}

// New message type for DeleteUser response
message DeleteUserResponse {
  bool success = 1;
}

message UsersResponse {
  repeated User users = 1;
}

service UsersService {
  rpc GetUsers (Empty) returns (UsersResponse);
  rpc CreateUser (CreateUserRequest) returns (User);
  // New RPC methods
  rpc GetUserById (GetUserByIdRequest) returns (User);
  rpc UpdateUser (UpdateUserRequest) returns (User);
  rpc DeleteUser (DeleteUserRequest) returns (DeleteUserResponse);
}

Now, update apps/users-service/src/app/app.controller.ts:

// apps/users-service/src/app/app.controller.ts
import { Controller } from '@nestjs/common';
import { GrpcMethod, ClientProxy, Inject, EventPattern, Payload } from '@nestjs/microservices';
import { AppService } from './app.service';

// Define the expected types for our gRPC messages (matching .proto)
interface Empty { /* no fields */ }
interface User { id: number; name: string; email: string; }
interface UsersResponse { users: User[]; }
interface CreateUserRequest { name: string; email: string; }
interface GetUserByIdRequest { id: number; } // New interface
interface UpdateUserRequest { id: number; name?: string; email?: string; } // New interface
interface DeleteUserRequest { id: number; } // New interface
interface DeleteUserResponse { success: boolean; } // New interface

// Define the expected type for the UserCreatedEvent payload
interface UserCreatedEvent {
  id: number;
  name: string;
  email: string;
}

@Controller()
export class AppController {
  constructor(
    private readonly appService: AppService,
    @Inject('RABBITMQ_SERVICE') private readonly rabbitmqClient: ClientProxy,
  ) {}

  // Existing gRPC method to get all users
  @GrpcMethod('UsersService', 'GetUsers')
  async getUsers(data: Empty): Promise<UsersResponse> {
    console.log('Users Microservice (gRPC) received request for GetUsers');
    const users = await this.appService.findAllUsers(); // Use TypeORM method
    return { users: users };
  }

  // Existing gRPC method to create a user and emit an event
  @GrpcMethod('UsersService', 'CreateUser')
  async createUser(@Payload() user: CreateUserRequest): Promise<User> {
    console.log(`Users Microservice (gRPC) received request to create user: ${user.name}`);
    const newUser = await this.appService.createUser(user.name, user.email); // Use TypeORM method

    this.rabbitmqClient.emit<UserCreatedEvent>('user_created', newUser);
    console.log(`[EVENT EMITTED] 'user_created' event emitted for user: ${newUser.name}`);

    return newUser;
  }

  // New gRPC method to get a user by ID
  @GrpcMethod('UsersService', 'GetUserById')
  async getUserById(@Payload() data: GetUserByIdRequest): Promise<User> {
    console.log(`Users Microservice (gRPC) received request for GetUserById: ${data.id}`);
    const user = await this.appService.findUserById(data.id);
    if (!user) {
      // In a real application, you'd throw a NotFoundException or similar gRPC error
      // For simplicity, we'll return an empty user or handle null
      console.warn(`User with ID ${data.id} not found.`);
      return { id: 0, name: '', email: '' }; // Return a default empty user
    }
    return user;
  }

  // New gRPC method to update a user
  @GrpcMethod('UsersService', 'UpdateUser')
  async updateUser(@Payload() data: UpdateUserRequest): Promise<User> {
    console.log(`Users Microservice (gRPC) received request to update user: ${data.id}`);
    const updatedUser = await this.appService.updateUser(data.id, data.name, data.email);
    if (!updatedUser) {
      console.warn(`User with ID ${data.id} not found for update.`);
      return { id: 0, name: '', email: '' }; // Return a default empty user
    }
    return updatedUser;
  }

  // New gRPC method to delete a user
  @GrpcMethod('UsersService', 'DeleteUser')
  async deleteUser(@Payload() data: DeleteUserRequest): Promise<DeleteUserResponse> {
    console.log(`Users Microservice (gRPC) received request to delete user: ${data.id}`);
    const success = await this.appService.deleteUser(data.id);
    return { success: success };
  }
}

Explanation:

• We've added GetUserById, UpdateUser, and DeleteUser gRPC methods, mapping them to the corresponding service methods.
• The @Payload() decorator automatically extracts the data from the incoming gRPC message.
• Error handling for "not found" cases is simplified for this tutorial; in a production system, you'd use NestJS's exception filters or gRPC-specific error handling mechanisms.

Updating the api-gateway to Use New CRUD Operations

Finally, our api-gateway needs new HTTP endpoints to trigger these new gRPC CRUD operations.

Step 1: Update api-gateway/src/app/app.controller.ts

// apps/api-gateway/src/app/app.controller.ts
import { Controller, Get, Post, Body, Inject, Param, Put, Delete, HttpCode, HttpStatus } from '@nestjs/common'; // Add Put, Delete, Param, HttpCode, HttpStatus
import { ClientProxy } from '@nestjs/microservices';
import { AppService } from './app.service';
import { Observable } from 'rxjs';
import { map } from 'rxjs/operators'; // Import map operator

// Define the expected types for our gRPC messages (matching .proto)
interface User { id: number; name: string; email: string; }
interface UsersResponse { users: User[]; }
interface CreateUserRequest { name: string; email: string; }
interface GetUserByIdRequest { id: number; }
interface UpdateUserRequest { id: number; name?: string; email?: string; }
interface DeleteUserRequest { id: number; }
interface DeleteUserResponse { success: boolean; }

@Controller('users') // Set a base path for user-related endpoints
export class AppController {
  constructor(
    private readonly appService: AppService,
    @Inject('USERS_SERVICE') private readonly usersServiceClient: ClientProxy,
  ) {}

  // Get all users
  @Get() // GET /users
  getUsers(): Observable<UsersResponse> {
    console.log('API Gateway received HTTP GET request for /users, forwarding to gRPC service');
    return this.usersServiceClient.send<UsersResponse>('GetUsers', {});
  }

  // Get user by ID
  @Get(':id') // GET /users/:id
  getUserById(@Param('id') id: string): Observable<User> {
    console.log(`API Gateway received HTTP GET request for /users/${id}, forwarding to gRPC service`);
    // Convert ID to number as expected by gRPC message
    return this.usersServiceClient.send<User>('GetUserById', { id: parseInt(id, 10) });
  }

  // Create a new user
  @Post() // POST /users
  createUser(@Body() user: CreateUserRequest): Observable<User> {
    console.log('API Gateway received HTTP POST request for /users, forwarding to gRPC service');
    return this.usersServiceClient.send<User>('CreateUser', user);
  }

  // Update an existing user
  @Put(':id') // PUT /users/:id
  updateUser(@Param('id') id: string, @Body() user: { name?: string; email?: string }): Observable<User> {
    console.log(`API Gateway received HTTP PUT request for /users/${id}, forwarding to gRPC service`);
    const updatePayload: UpdateUserRequest = {
      id: parseInt(id, 10),
      name: user.name,
      email: user.email,
    };
    return this.usersServiceClient.send<User>('UpdateUser', updatePayload);
  }

  // Delete a user
  @Delete(':id') // DELETE /users/:id
  @HttpCode(HttpStatus.NO_CONTENT) // Return 204 No Content on successful deletion
  deleteUser(@Param('id') id: string): Observable<void> {
    console.log(`API Gateway received HTTP DELETE request for /users/${id}, forwarding to gRPC service`);
    // Send the delete request and map the response to void (no content)
    return this.usersServiceClient.send<DeleteUserResponse>('DeleteUser', { id: parseInt(id, 10) }).pipe(
      map(() => undefined) // Map the response to void for 204 No Content
    );
  }

  // Original /hello endpoint (if you kept it)
  // @Get() // This conflicts with @Get() on the controller, remove or rename
  // getData() {
  //   return this.appService.getData();
  // }
}

Important Note on @Controller('users'): I've added @Controller('users') to the AppController. This means all routes within this controller will be prefixed with /users. So, getUsers() becomes GET /users, getUserById(':id') becomes GET /users/:id, etc. This is a common and clean way to organize RESTful APIs. If you still have the getData() method from previous parts, you might need to remove it or move it to a different controller to avoid route conflicts.

Explanation:

• New HTTP methods: @Get(':id'), @Post(), @Put(':id'), @Delete(':id') are used to map to standard RESTful CRUD operations.
• @Param('id'): Extracts the id from the URL path.
• @Body(): Extracts the request body.
• parseInt(id, 10): Converts the string ID from the URL to a number, as expected by our Protobuf messages.
• @HttpCode(HttpStatus.NO_CONTENT): For DELETE operations, it's common practice to return a 204 No Content status code on success, indicating the resource was deleted and there's no body to return.
• .pipe(map(() => undefined)): Used with deleteUser to transform the Observable<DeleteUserResponse> into Observable<void>, which NestJS interprets as a signal to send a 204 No Content response.

Time to Test Our Full CRUD!

You'll need four separate terminal windows open at the root of your nestjs-ms-blueprint monorepo (plus your Docker container running).

1. Terminal 1 (for Docker PostgreSQL): Ensure it's running (docker ps).

2. Terminal 2 (for users-service):

   nx serve users-service
   
   

   Wait for Users Microservice (gRPC) is listening on 127.0.0.1:3001.

3. Terminal 3 (for notifications-service):

   nx serve notifications-service
   
   

   Wait for Notifications Microservice (RabbitMQ) is listening for user_events_queue.

4. Terminal 4 (for api-gateway):

   nx serve api-gateway
   
   

   Wait for 🚀 Application is running on: http://localhost:3000/api.

Now, open Postman or Insomnia and perform the following sequence of requests:

1. Create a User (POST)

• Method: POST
• URL: http://localhost:3000/users

• Body (raw JSON):

  {
    "name": "John Doe",
    "email": "john.doe@example.com"
  }
  
  

• Expected Response: 201 Created with the created user object (including an id).
• Terminal Logs: Observe logs in api-gateway, users-service (DB save, event emit), and notifications-service (email sent).

2. Create Another User (POST)

• Method: POST
• URL: http://localhost:3000/users

• Body (raw JSON):

  {
    "name": "Jane Smith",
    "email": "jane.smith@example.com"
  }
  
  

• Expected Response: 201 Created with the new user object.

3. Get All Users (GET)

• Method: GET
• URL: http://localhost:3000/users
• Expected Response: 200 OK with an array containing both John Doe and Jane Smith.

4. Get User by ID (GET)

• Method: GET
• URL: http://localhost:3000/users/1 (replace 1 with the actual ID of John Doe from step 1)
• Expected Response: 200 OK with John Doe's user object.

5. Update User (PUT)

• Method: PUT
• URL: http://localhost:3000/users/1 (replace 1 with John Doe's ID)

• Body (raw JSON):

  {
    "name": "Jonathan Doe",
    "email": "jonathan.doe@example.com"
  }
  
  

• Expected Response: 200 OK with the updated user object.

6. Delete User (DELETE)

• Method: DELETE
• URL: http://localhost:3000/users/2 (replace 2 with Jane Smith's ID)
• Expected Response: 204 No Content.

7. Verify Deletion (GET All Users)

• Method: GET
• URL: http://localhost:3000/users
• Expected Response: 200 OK with only Jonathan Doe remaining in the array.

If all these steps work, you've successfully integrated a PostgreSQL database with TypeORM into your users-service and exposed full CRUD functionality through your API Gateway using gRPC!

Wrapping Up Part 5

Incredible work today! You've tackled one of the most critical and often misunderstood aspects of microservices: data management. You've moved beyond mock data and integrated a real PostgreSQL database using TypeORM, enabling your users-service to persist and manage user information effectively.

Key takeaways from this part:

• The importance of the "database per service" principle for true microservice autonomy.
• How to quickly set up a PostgreSQL database using Docker.
• Integrating TypeORM into your NestJS service, defining entities, and using the Repository for CRUD operations.
• Extending your gRPC API to expose these CRUD functionalities.
• Updating your API Gateway to consume these new gRPC methods via standard HTTP endpoints.

You now have a users-service that is a fully capable data owner, managing its own data store and exposing that data through a well-defined gRPC API. This is a cornerstone of a robust microservice architecture.

Next up, we'll tackle security. How do we ensure that only authenticated and authorized users can access our precious data?

Get ready for Part 6: Who Are You?: Implementing JWT Authentication in Your API Gateway! We'll secure our API Gateway and learn how to pass user context down to our services.

See you there!