Using GORM with Golang Micro-services and PostgreSQL
Micro-services importance
In modern software development, micro-services architecture is key for creating scalable and maintainable applications. It breaks complex apps into smaller, independent services, letting teams develop, deploy, and scale parts separately. This increases agility and resilience.
However, with this architectural shift comes the challenge of managing data across these distributed services. Efficient database interaction is crucial to ensure that micro-services perform optimally. This is where GORM, a powerful Object-Relational Mapper (ORM) for Go, comes into play.
This post shows how to use GORM with Golang micro-services and PostgreSQL. We'll give practical examples and best practices to boost development speed, scalability, and maintainability. Whether you're new to micro-services or want to improve your current setup, this guide will help you make the most of GORM, Golang, and PostgreSQL in modern software development.
To make it easier to follow this guide, please review the code that was developed here
What is ORM?
Imagine you have two worlds: in one of these worlds is your application code, there exist objects, classes, and methods; and the other world is the databases world, where your data is stored in tables with rows and columns. These worlds speak different languages and have different ways of organizing information.
An ORM acts as a translator between these two worlds. It allows us to interact with our database using the language and syntax of your programming language. Instead of writing complex SQL queries and manually mapping results to objects in your code, you can define classes and structures that represent tables and rows in your database.
GORM
In Go, we have the GORM package, an ORM library designed to be developer-friendly. (see
GORM simplifies database interactions by providing an easy-to-use API for CRUD operations, complex queries, and transaction management, all while abstracting much of the boilerplate code.
First steps with GORM
This guide assumes that the reader has basic knowledge of Go and micro-services. It also assumes that you have Go installed. It is not intended to be a basic guide to these topics.
Creating a new Golang project
The first step is to create a new project in Go:
go mod init <name>I created a project called stellar_backend. This command generates two essential files: go.mod and go.sum. These files are critical components of the Go module system, managing dependencies in Go projects.
Next, we need to install GORM and the PostgreSQL driver, which can be done using the go get command
go get -u gorm.io/gorm
go get -u gorm.io/driver/postgresSince we created a module, these commands will update our go.sum and go.mod files to include the necessary dependencies. As we continue, we will see other dependencies to install, pay attention to those that are not on your system to install them using the go get command.
Structuring project for scalability
A well-structured file layout is essential for maintainability and scalability. I use the following layout to structure my micro-services in Go, it helps me keep the code readable and modularized.
├── api/
│ │
│ └── server/
│ └── main.go
│
├── config/
│ └── config.go
├── internal/
│ ├── <module>/
│ │ └── service/
│ │ │ └── <module>_service.go
│ │ └── repositories/
│ │ │ └── <module>_repository.go
│ │ └── dtos/
│ │ │ └── <module>_dto.go
│ │ └── controller.go
│ └── db/
│ └── db.go
├── .env
├── go.mod
└── go.sumSummary of the most important parts:
api/:
server/main.go: It contains the entry point of our application.
internal/: Contains application-specific internal logic.
db/
db/db.go Contains data base connection logic.
<module_name>/:
repositories/<module_name>_repository.go: Manages database interactions using GORM.
dtos/<module_name>_dto.go: Define module DTOs. (Data Transfer Object).
routes.go: Defines an API to manage module operations and registers module paths on the server.
go.mod: The module definition file created by go mod init.
go.sum: The dependency checksum file managed by Go modules.
.env: Contains the environmental variables.
Before proceeding with the code, we need to set up a database for our queries
Creating a database
Connecting to a database is paramount. You can either create a new database or use an existing one. I'll use Docker and Docker Compose to set up and manage a database in a container.
So, I created a docker-compose.yml file with this content:
...
services:
db:
...
environment:
POSTGRES_PASSWORD: $POSTGRES_PASSWORD
POSTGRES_DB: $POSTGRES_DB
...
networks:
default:
name: stellar_defaultAnd, in the .env file I defined the variables that are used in this file:
POSTGRES_DB=stellar_backend
POSTGRES_PASSWORD=admin
POSTGRES_USER=adminSo now, we can run the command docker-compose up to launch the container and create the database. We should have an output similar to the following:
Now, let’s try to connect to this new DB using a postgreSQL client, I used psql to do that:
psql -h localhost -U admin -d stellar_backendIt will prompt you for the password (which we previously defined in the .env file) for your user. If the connection is successful, you will see something like this:
Now, let's connect to our database using Go.
Connecting GORM with the database
Now that we have our database set up, we need to connect to it using Go. Following our project structure, let's create a db.go file to handle the connection logic, in that file let’s put our connection to db logic:
...
dsn := "host=" + os.Getenv("POSTGRES_HOST") +
" user=" + os.Getenv("POSTGRES_USER") +
" password=" + os.Getenv("POSTGRES_PASSWORD") +
" dbname=" + os.Getenv("POSTGRES_DB") +
" port=" + os.Getenv("POSTGRES_PORT") +
" sslmode=" + os.Getenv("SSL_MODE") +
" TimeZone=" + os.Getenv("TIMEZONE")
db, err = gorm.Open(postgres.Open(dsn), &gorm.Config{})
...
fmt.Println("Connected to database! 🚀")
...For now, the file structure seems like this:
my-microservice/
├── internal/
│ └── db/
│ └── db.go
├── .env
├── docker-compose.yml
├── go.mod
└── go.sumAnd we need to add the new variables to the .env file:
...
POSTGRES_PORT=5432
POSTGRES_DB=stellar_db
SSL_MODE=disable
TIMEZONE=UTCAnd in order to load those variables we need the package github.com/joho/godotenv. To install it just run: go get -u github.com/joho/godotenv
If we run this code and everything goes smoothly, we'll see the following message:
Connected to database! 🚀The next step is to create a table to store data, for that we will need models.
Creating a model
A model represent a structured definition of a database table, serves as a blueprint for interacting with it, and provides the way to define, manipulate and query data using our programming language, in this case Go.
The models in Go and GORM are defined in this way:
type <model_name> struct {
<property_name> <type> <struct_tags>
}In Go, we use a struct to define a model. Each struct represents a database table, and the fields in the struct match the columns in that table. struct fields can also show relationships between tables. Each field has a specific data type and can include tags for extra attributes or rules.
For example, I will create a model to represent planetary systems. My fictional micro-service will provide an API to handle information about these systems (all fictional).
...
type PlanetarySystem struct {
gorm.Model
Name string `gorm:"not null;type:varchar(100)" json:"name"`
Planets []Planet `gorm:"foreignKey:PlanetarySystemID" json:"planets"`
Asteroids []Asteroid `gorm:"foreignKey:PlanetarySystemID" json:"asteroids"`
}
...Just to take an practical example, my PlanetarySystem model has the following properties:
Name: It will be the name of the planetary system.
The struct tag
gorm:"not null;type:varchar(100)"specifies that this field cannot be null in the database and has a maximum length of 100 characters.
Planets: It will be an array that stores the planets that belong to the system.
The struct tag
gorm:"foreignKey:PlanetarySystemID"indicates that there is a foreign key relationship between thePlanetarySystemandPlanetmodels. Specifically, it tells GORM that thePlanetmodel has aPlanetarySystemIDfield which acts as a foreign key to associate each planet with a planetary system.
Asteroids: It will be an array that stores the asteroids in the system.
Similar to the
Planetsfield.
The json tags in the struct definitions specify the field names to be used when encoding or decoding JSON data. For example, json:"name" ensures that the Name field is represented as "name" in the JSON outpu
The models come with a basic Go struct “gorm.Model”, which includes some basics properties like ID, CreatedAt, UpdatedAt and DeletedAt, which will be useful for managing the models.
You can learn more about how to declare models here
This on an ER diagram could be represented as follows:
Running the migrations
Now, in order to make the tables in the DB using those models we must run the migrations. A migration is a way to manage and apply changes to the db schema, to keep it sync with the models. In db.go we need to call the db.Automigrate function. So let’s add it:
internal/db/db.go
...
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
panic("failed to connect database")
}
fmt.Println("Connected to database! 🚀")
db.AutoMigrate(
&models.PlanetarySystem{},
&models.Planet{},
...other models
)
...
After having run this code, we should have the tables already created. You can check if the tables were created using psql and the command \dt:
stellar_backend-# \dt
List of relations
Schema | Name | Type | Owner
--------+-------------------+-------+-------
public | asteroids | table | admin
public | astronauts | table | admin
...
(5 rows)We can check details about the tables using \d+ and the name of the table.
So what's next? Well, those tables are empty, we want to be able to add, read, edit and remove data in those tables. To do this, we need to create the HTTP server, routes, and controllers.
Set Up an HTTP Server
Creating HTTP Server
I used echo framework to set up an HTTP server to serve my endpoints
api/server/main.go
It contains the entry point of our program and it starts the server that is defined in server.go
import "stellar_backend/internal/server"
...
func main() {
server.Server.Start(":8080")
}
...internal/server/server.go
This code has an init function where a new server and a database instance are created, and it calls InitRoutes function for all the modules that will have endpoints that the server needs to serve. For the server, I used the Echo web framework.
var Server *echo.Echo
func init() {
Server = echo.New()
db := db.DB()
planet.InitRoutes(Server, db)
...
}routes.go
For each module that requires endpoints I like to have a routes.go file to contain the handler of each endpoint and its InitRoutes function, similar to this one:
...
func InitRoutes(server *echo.Echo, db *gorm.DB) {
server.GET("/planetarysystems", getPlanetarySystems(db))
...
}
func getPlanetarySystems(db *gorm.DB) echo.HandlerFunc {
return func(c echo.Context) error {
return c.JSON(200, "Get Planetary Systems")
}
}
...The InitRoutes function sets up HTTP routes on the provided Echo server. It maps specific endpoints to their respective handler functions for managing the model. Each route (GET, POST, PUT, DELETE) is associated with a function that processes requests and interacts with the database through GORM.
Our project now seems like this:
├── api/
│ └── server/
│ └── main.go
│
├── internal/
│ ├── planetarySystem/
│ │ └── routes.go
...
│ ├── server/
│ │ └── server.go
│ └── db/
│ └── db.go
│
├── ...
└── go.sumRunning the server
Prerequisites:
Get up the PostgreSQL container using
docker-compose up
Run the code:
go run ./api/server/main.goIn order to improve the workflow I use a Makefile, so by using make build and make start , I can automate the complete build process.
The output should be similar to this one:
Processing a request
For the planetarySystems module, my routes.go code has the functions to process the requests; For example, here is an example to retrieve a message when a GET reaches the /planetarysystems endpoint:
internal/planetarySystems/routes.go
...
server.GET("/planetarysystems", getPlanetarySystems(db))
...
func getPlanetarySystems(db *gorm.DB) echo.HandlerFunc {
return func(c echo.Context) error {
return c.JSON(200, "Get Planetary Systems")
}
}So if I access to the route http://localhost:3000/planetarysystems I get the message:
First request response
Now that everything is set up and working, it's time to implement basic CRUD operations for our models to manage the data in the database.
Implementing basic CRUD
To add the CRUD operations, I updated the routes.go file for each model. I used Dependency Injection to create a decoupled controller, so the updated code for my models looks like this:
internal/planetarySystem/routes.go
...
func InitRoutes(server *echo.Echo, db *gorm.DB) {
controller := NewPlanetarysystemController(db)
server.GET("/planetarysystems", controller.getPlanetarySystems)
...
}
type planetarysystemController struct {
repository repository.PlanetarysystemRepository
}
type PlanetarysystemController interface {
getPlanetarySystems(c echo.Context) error
...
}
func NewPlanetarysystemController(db *gorm.DB) PlanetarysystemController {
return &planetarysystemController{
repository: repository.NewPlanetarysystemRepository(db),
}
}
func (controller *planetarysystemController) getPlanetarySystems(c echo.Context) error {
systems, err := controller.repository.GetPlanetarySystems()
if err != nil {
return c.JSON(500, err)
}
return c.JSON(200, systems)
}
...I decided to create a repository for each model to encapsulating all the data access operations and to abstract it from the business logic layer.
For example, the repository code for the PlanetarySystems module is the following:
...
type PlanetarysystemRepository interface {
GetPlanetarySystems() ([]models.PlanetarySystem, error)
SavePlanetarySystem(*models.PlanetarySystem) error
...
}
...
func (r *planetarysystemRepository) GetPlanetarySystems() ([]models.PlanetarySystem, error) {
var planetarySystems []models.PlanetarySystem
if err := r.db.Find(&planetarySystems).Error; err != nil {
return nil, err
}
return planetarySystems, nil
}
func (r *planetarysystemRepository) SavePlanetarySystem(planetarySystem *models.PlanetarySystem) error {
if err := r.db.Create(planetarySystem).Error; err != nil {
return err
}
return nil
}
...In this code we have a NewPlanetarysystemRepository function that is a constructor function that creates and returns an instance of planetarysystemRepository, initialized with the provided GORM database connection.
You can notice that I implemented 4 functions for the PlanetarysystemRepository interface :
GetPlanetarySystems: Get all the Planetary Systems from database.SavePlanetarySystem: Create or Update a Planetary System in database.GetPlanetarySystem: Get information about a Planetary System based on its IDDeletePlanetarySystem: Removes a Planetary System based on its ID.
There are the functions we need to implement the basic CRUD operations. For the other modules you can use the same patterns and code style we used for this specific module and we can complete the API.
Testing
Now we can test the endpoints using Postman.
Creating an entry
We can check if the table was updated to include this new entry:
Getting the entries:
In this case, the application performs the reverse operation: it retrieves data from the database using the model and then converts it to JSON format for transmission.
Removing an entry
You can check that the deleted_at column was updated with the timestamp of when the request was sent. This is called soft delete.
Summary
In this post, we've taken a comprehensive journey through integrating GORM into a Go micro-service, covering the essentials from understanding the significance of micro-services and ORMs to hands-on implementation.
What’s next?
Now that you've integrated GORM with Go micro-services, here are a few next steps to deepen your expertise:
Explore Advanced GORM Features: Delve into transactions, hooks, and associations for more complex database operations.
Optimize Performance: Learn about indexing, query optimization, and caching to enhance application performance.
Enhance Security: Implement best practices for authentication, authorization, and data encryption.
Automate with CI/CD: Set up Continuous Integration and Continuous Deployment pipelines to streamline your development workflow.
Join the Community: Engage with the Go and GORM communities, contribute to open-source projects, and stay updated with the latest trends.
By pursuing these next steps, you'll continue to build on your knowledge and create even more robust and efficient micro-services. Happy coding!
Partner with Golang Experts
Let's discuss how our Golang expertise can elevate your next project. Connect with us and let's make it a success together
