moleculer-db

Moleculer service to store entities in database.

Features

• default CRUD actions
• cached actions
• pagination support
• pluggable adapter (NeDB is the default memory adapter for testing & prototyping)
• official adapters for MongoDB, PostgreSQL, SQLite, MySQL, MSSQL.
• fields filtering
• populating
• encode/decode IDs
• entity lifecycle events for notifications

Install

$ npm install moleculer-db --save

Usage

"use strict";

const { ServiceBroker } = require("moleculer");
const DbService = require("moleculer-db");

const broker = new ServiceBroker();

// Create a DB service for `user` entities
broker.createService({
    name: "users",
    mixins: [DbService],

    settings: {
        fields: ["_id", "username", "name"]
    },

    afterConnected() {
        // Seed the DB with ˙this.create`
    }
});

broker.start()

// Create a new user
.then(() => broker.call("users.create", {
    username: "john",
    name: "John Doe",
    status: 1
}))

// Get all users
.then(() => broker.call("users.find").then(console.log));

// List users with pagination
.then(() => broker.call("users.list", { page: 2, pageSize: 10 }).then(console.log));

// Get a user
.then(() => broker.call("users.get", { id: 2 }).then(console.log));

// Update a user
.then(() => broker.call("users.update", { id: 2, name: "Jane Doe" }).then(console.log));

// Delete a user
.then(() => broker.call("users.remove", { id: 2 }).then(console.log));

Settings

Note: idField does not work with Sequelize adapter as you can freely set your own ID while creating the model.

Actions

find

Find entities by query.

Parameters

Results

Type: Array.<Object>

List of found entities.

count

Get count of entities by query.

Parameters

Results

Type: Number

Count of found entities.

list

List entities by filters and pagination results.

Parameters

Results

Type: Object

List of found entities and count with pagination info.

create

Create a new entity.

Parameters

Results

Type: Object

Saved entity.

insert

Create many new entities.

Parameters

Results

Type: Object, Array.<Object>

Saved entity(ies).

get

Get entity by ID.

Parameters

Results

Type: Object, Array.<Object>

Found entity(ies).

update

Update an entity by ID.

After update, clear the cache & call lifecycle events.

Parameters

Results

Type: Object

Updated entity.

remove

Remove an entity by ID.

Parameters

Results

Type: Number

Count of removed entities.

Methods

sanitizeParams

Sanitize context parameters at find action.

Parameters

Results

Type: Object

getById

Get entity(ies) by ID(s).

Parameters

Results

Type: Object, Array.<Object>

Found entity(ies).

entityChanged

Clear the cache & call entity lifecycle events

Parameters

Results

Type: Promise

clearCache

Clear cached entities

Parameters

Results

Type: Promise

transformDocuments

Transform the fetched documents

Parameters

Results

Type: Array, Object

validateEntity

Validate an entity by validator.

Parameters

Results

Type: Promise

encodeID

Encode ID of entity.

Parameters

Results

Type: any

decodeID

Decode ID of entity.

Parameters

Results

Type: any

_find

Find entities by query.

Parameters

Results

Type: Array.<Object>

List of found entities.

_count

Get count of entities by query.

Parameters

Results

Type: Number

Count of found entities.

_list

List entities by filters and pagination results.

Parameters

Results

Type: Object

List of found entities and count.

_create

Create a new entity.

Parameters

Results

Type: Object

Saved entity.

_insert

Create many new entities.

Parameters

Results

Type: Object, Array.<Object>

Saved entity(ies).

_get

Get entity by ID.

Parameters

Results

Type: Object, Array.<Object>

Found entity(ies).

_update

Update an entity by ID.

After update, clear the cache & call lifecycle events.

Parameters

Results

Type: Object

Updated entity.

_remove

Remove an entity by ID.

Parameters

Populating

The service supports to populate fields from other services. E.g.: if you have an author field in post entity, you can populate it with users service by ID of author. If the field is an Array of IDs, it will populate all entities via only one request.

Example of populate schema

broker.createService({
    name: "posts",
    mixins: [DbService],
    settings: {
        populates: {
            // Shorthand populate rule. Resolve the `voters` values with `users.get` action.
            "voters": "users.get",

            // Define the params of action call. It will receive only with username & full name of author.
            "author": {
                action: "users.get",
                params: {
                    fields: "username fullName"
                }
            },

            // Custom populator handler function
            "rate"(ids, docs, rule, ctx) {
                return Promise.resolve(...);
            }
        }
    }
});

// List posts with populated authors
broker.call("posts.find", { populate: ["author"]}).then(console.log);

The populate parameter is available in find, list and get actions.

Lifecycle entity events

There are 3 lifecycle entity events which are called when entities are manipulated.

broker.createService({
    name: "posts",
    mixins: [DbService],
    settings: {},

    afterConnected() {
        this.logger.info("Connected successfully");
    },

    entityCreated(json, ctx) {
        this.logger.info("New entity created!");
    },

    entityUpdated(json, ctx) {
        // You can also access to Context
        this.logger.info(`Entity updated by '${ctx.meta.user.name}' user!`);
    },

    entityRemoved(json, ctx) {
        this.logger.info("Entity removed", json);
    },    
});

Please note! If you manipulate multiple entities (updateMany, removeMany), the json parameter will be a Number instead of entities!

Extend with custom actions

Naturally you can extend this service with your custom actions.

const DbService = require("moleculer-db");

module.exports = {
    name: "posts",
    mixins: [DbService],

    settings: {
        fields: ["_id", "title", "content", "votes"]
    },

    actions: {
        // Increment `votes` field by post ID
        vote(ctx) {
            return this.adapter.updateById(ctx.params.id, { $inc: { votes: 1 } });
        },

        // List posts of an author
        byAuthors(ctx) {
            return this.find({
                query: {
                    author: ctx.params.authorID
                },
                limit: ctx.params.limit || 10,
                sort: "-createdAt"
            });
        }
    }
}

Remove default actions

According to moleculer documentation you can disable an action when override it with false

const DbService = require("moleculer-db");

module.exports = {
    name: "posts",
    mixins: [DbService],

    actions: {
        // Disable find default action
        find: false
    }
}

Test

$ npm test

In development with watching

$ npm run ci

License

The project is available under the MIT license.

Contact

Copyright (c) 2016-2024 MoleculerJS

0.8.23 (2023-02-19)

Changes

• fix issues with exclude fields #349 #350

0.8.22 (2023-02-11)

Changes

• if param.fields is null, use this.settings.fields directly #340
• split improve #339
• new settings.excludeFields feature to exclude fields #341

0.8.21 (2022-12-17)

Changes

• Fix bug - populate not working - idField mapped to undefined #327
• moleculer-db-adapter-mongoose crashing if in state "connecting" fix #329
• Recursive deep populate #331
• beforeEntity(Change|Update|Remove) lifecycle events #330
• fix authorize fields unsafe check #335

0.8.18 (2022-05-25)

Changes

• upgrade seald-io/nedb to 3.0.0 #315
• async custom validation #306
• fix count just the right number of row #304
• fix RegExp throw error if search startsWith '.' or '+' #316

0.8.13 (2021-04-26)

Changes

• add cacheCleanEventType service setting #258

0.8.12 (2021-02-15)

Changes

• support custom NeDB Datastores (in MemoryDbAdapter contructor) #248

0.8.11 (2020-11-16)

Changes

• fix missing limit when directly call this._list #229

0.8.10 (2020-09-27)

Changes

• allow to specify resulting populated field name #224
• update typescript definitions #202, #208

0.8.8 (2020-06-06)

Changes

• handling nested fields in populating. #177
• use dot notation in update action in case of useDotNotation: true. #183

0.8.7 (2020-04-08)

Changes

• fix mapping in get action with simple ID. #176

0.8.5 (2020-02-06)

Changes

• fix fields property issue in update action. #152

0.8.4 (2019-11-19)

Changes

• parsing query string property in find & list actions by @jjgumucio. PR #149

0.8.1 (2019-07-07)

Changes

• update dependencies
• add rest properties for CRUD actions

0.8.0 (2019-07-01)

Changes

• add new CRUD (_find, _count, _insert, _create, _get, _update, _remove, ) methods by @rzvdaniel

0.7.5 (2018-07-13)

Changes

• fix custom entityValidator calling

0.7.0 (2017-11-13)

Breaking changes

• transformDocuments and entityChanged calls have been moved from methods to actions.
• this.find(ctx, params) method has been removed. Use this.adapter.find(params) instead.
• this.count(ctx, params) method has been removed. Use this.adapter.count(params) instead.
• this.create(ctx, params) method has been removed. Use this.adapter.insert(params.entity) instead.
• this.createMany(ctx, params) method has been removed. Use this.adapter.insertMany(params.entities) instead.
• this.getById(ctx, params) arguments have been changed. First argument is the ID or IDs, second argument is a decoding boolean. If true, it calls the decodeID method with every ID. The mapping feature has been moved to get action.
• this.updateById(ctx, params) method has been removed. Use this.adapter.updateById(params.id, params.update) instead.
• this.updateMany(ctx, params) method has been removed. Use this.adapter.updateMany(params.query, params.update) instead.
• this.removeById(ctx, params) method has been removed. Use this.adapter.removeById(params.id) instead.
• this.removeMany(ctx, params) method has been removed. Use this.adapter.removeMany(params.query) instead.
• this.clear(ctx) method has been removed. Use this.adapter.clear() instead and call the entityChanged method.

New

• adapters have a new findOne method which returns with only one entity by query.

0.6.4 (2017-11-12)

Changes

• add findOne method to get one record by a query

0.6.0 (2017-09-12)

Breaking changes

• update Moleculer to v0.11.x

0.5.0 (2017-08-08)

Changes

Changed create & insert actions (breaking)

The current create action renamed to insert. It accepts entity param to create an entity. Or entities array to create multiple entities. The create actions save a new entity from the params directly. So you can send an entity JSON directly to the create action.

// Create a new entity
broker.call("users.create", {
    name: "John",
    age: 33,
    status: true
});

// Create a new entity
broker.call("users.insert", {
    entity: {
        name: "John",
        age: 33,
        status: true
    }
});

// Create multiple entities
broker.call("users.insert", {
    entities: [
        {
            name: "John",
            age: 33,
            status: true
        },
        {
            name: "Jane",
            age: 28,
            status: true
        }
    ]
});

Better update action (breaking)

The update action update entity fields from params directly. You don't need to wrap it under an update prop.

broker.call("users.update", {
    id: 5,
    name: "Jane",
    status: false
});

Minor changes

• added EntityNotFoundError.

0.4.0 (2017-07-17)

New

Encoding & decoding IDs

There are two new encodeID and decodeID methods. You can use them if you want to encode & decode database ID (for example with hashids)

const Hashids = require("hashids");
const hashids = new Hashids("secret salt");

broker.createService({
    name: "posts",
    mixins: [DbService],
    methods: {
        encodeID(id) {
            return hashids.encodeHex(id);
        },
        decodeID(id) {
            return hashids.decodeHex(id);
        }
    }
});

Entity lifecycle events

There are 3 entity lifecycle events which are called when entities are manipulated.

broker.createService({
    name: "posts",
    mixins: [DbService],
    settings: {},

    afterConnected() {
        this.logger.info("Connected successfully");
    },

    entityCreated(json, ctx) {
        this.logger.info("New entity created!");
    },

    entityUpdated(json, ctx) {
        // You can also access to Context
        this.logger.info(`Entity updated by '${ctx.meta.user.name}' user!`);
    },

    entityRemoved(json, ctx) {
        this.logger.info("Entity removed", json);
    },
});

Better fields filtering

A new fields filtering method is implemented. It can also handle nested properties.

const DbService = require("moleculer-db");

module.exports = {
    name: "users",
    mixins: [DbService],

    settings: {
        fields: ["name", "address.city", "address.country", "bio"]
    }
}

broker.call("users.get", { id: 5, fields: ["name", "address", "bio.height", "bio.hair", "password"] }).then(console.log);
/* The returned object contains only the following fields:
{
    name: "Walter",
    address: {
        city: "Albuquerque",
        country: "USA"
    },
    bio: {
        height: 185,
        hair: "bald"
    }
}
*/

Changes

• deprecated fields as space-separated String in settings. Enabled only Array<String>.

• deprecated fields as space-separated String in fields of settings.populates. Enabled only Array<String>.

• BREAKING: model action & method is removed! Use get action instead.

• moleculer-db-adapter-mongoose returns with Mongoose objects instead of native object. But it will be converted to native JS object in [moleculer-db].

        customAction(ctx) {
            return this.adapter.find({}).then(docs => {
                // You can access the Mongoose virtual methods & getters of `docs` here
            });
        }

0.3.0 (2017-07-07)

New

New createMany method

A new createMany method is created. With it you can insert many entities to the database.

this.createMany(ctx, {
    entities: [...]
});

New list action with pagination

There is a new list action with pagination support.

broker.call("posts.list", { page: 2, pageSize: 10});

The result is similar as

{
    rows: [
        { title: 'Post #26' },
        { title: 'Post #27' },
        { title: 'Post #25' },
        { title: 'Post #21' },
        { title: 'Post #28' }
    ],
    total: 28,
    page: 2,
    pageSize: 10,
    totalPages: 3
}

New settings

• pageSize - Default page size in list action. Default: 10
• maxPageSize - Maximum page size in list action. Default: 100
• maxLimit - Maximum value of limit in find action. Default: -1 (no limit)

0.2.0 (2017-07-06)

Breaking changes

Renamed service methods

• findAll renamed to find
• update renamed to updateMany
• remove renamed to removeMany

clear action is removed

We removed the clear action from service The reason is if you don't filter it in whitelists of API gw, it will be published and callable from client-side what is very dangerous.

After all if you need it:

module.exports = {
    name: "posts",
    mixins: [DbService],

    actions: {
        clear(ctx) {
            return this.clear(ctx);
        }
    }
}

Renamed adapter methods

• findAll renamed to find
• update renamed to updateMany
• remove renamed to removeMany