Web Api Service- Part 2

We have the initial setup for our web api. All we need to do is adding the mongo conectivity along with some setup on MongoDB.

MongoDB limitation

Now before we are going to dig in the rest of coding we should now know some important limitations of MongoDB that will need to be taken into consideration.

  • Transactions exists only on document level. In SQL, you have the concept of a transaction that can involve multiple database objects(tables, cursors, view etc.), meaning you can acquire a lock, do some operations on many db objects, commit, or rollback depending on the output, on those db objects. More or less, in mongo this concept applies only to a document i.e one record to a collection that you are working on. So for example if you have the objects/documents  Student:
    {_id: 1, Name:'Jeff', Surname:'Andreson', Classes:['Algebra','Formal Languages','Calculus']},
    {_id: 2, Name:'Patty', Surname:'Andreson', Classes:['Formal Languages','Calculus']}
    

    and you start to edit them, once you will the first document, that document will be persisted on the collection, if the edit on the second document fails the rollback will be for that document and only that one, not like in sql, where a transaction rollback would be done on the all objects of the db that were affected from the beginning of the transaction. The subject is in more detail explained in MongoDB documentation. This means that for having all the benefits of this mechanism we should design the document to cover our current use cases, but this will have also a limitation see next point.

  • The size of a document can not exceed 16MB. There is a possibility of using GridFS, but also we need tot take into consideration the GridFS driver support. In this particular case I recommend a redesign of the structure of the document.

These are the most important ones, that I will take into consideration for more a thoroughly documentation i recommended the official documentation.

Now knowing these limitations, can make us reconsider Mongo, but, for our case, is perfect. We want to store last known location of a client that communicates with our app. If the client is sending updates every time it’s location has changed(passed a defined threshold), than we have a better understanding of their movement in time, if the client, can not send their new position is because:

  • Client does not have a internet connection.
  • Our app have crashed
  • Invalid input
  • Any of those reason does not impose the need of a transaction system. It would be nice, that all it’s locations that could not be sent due to no internet connection, should be store locally on the client, and once a internet connection is available, to be sent to our server, but that is beyond of our current scope.

Application design

In the first part we defined the routes:

  • Get method:api/v1/client/{clientId}/location: This will retrieve the last known location. If there is a time frame defined we will look into that also, this means that we can we have URL like:api/v1/client/{clientId}/location?timeStart=currentTimeStart&timeEnd=endTimedefined . Both of this requests will return an array of ClientLocation type defined below.
  • Post method:api/v1/client/{clientId}/location.  This will store the new location of the client. This route will get from request body the following infos:latitude,longitude,gps-time. This should be enough for the current use.

After the routes were refined, we will define the classes that will be used in the whole process. That means, we will have LocationRequest that will be used in the post method and should like this:

Along with that we will have a LocationReponse that will look like this:

Also the ClientLocation collection that will memorize all the locations of the clients inside mongo will have the following definition:

Coding the way

We have defined the domain, now we need to write the mongo logic for inserting and retrieving, along with some validations. In beginning of the series,I said that http,will be the way of communicating for clients location update and signalr will refresh the map with the latest location updates. We can also use signalr for clients location update, but this will imply clients knowing to “speak” signalr, we will present that later on. Right now we will create a new class LocationManager, that will encapsulate the mongo context, and will have 2 methods, RetrieveLocations and UpdateLocation we will use async api for performance and best practice reason. We can extract an interface and abstract all the way, but being a proof of concept I prefer to do it as it is, also the mongo context, can be mocked so, any unit test that you want to create it, it’s fairly easy to do.

First we are going to define a small interface, just for expressing the intetions:

Mainly we have the update location which essentialy will insert a new document in ClientLocation collection and RetrieveLocations method which will get us locations of a particular clientId, within a timeframe, if no timeframe defined we will get all.

Now let’st write the implementation of ILocationManager, LocationManager. At first we will create a MongoClient and from there we will extend as we are need to do our work as it shown in the next listing:

Let’s break it line by line. At first we are creating a MongoClient, this is the base class for mongo working, the mongodb connection string is like an URL :

mongodb://[user:passw]host:[port1]

Next we are retrieving the database that we are going to work with. Notice that this database will need to be created more details that at end. Afterwards we are exposing the collection which will be used for querying and inserting data. These properties are exposed as lambdas, also notice there are interface exposed which means that unit test is not as difficult as it first would appear. You may ask yourself where is the disposal part, the beautiful thing is that you don’t need to do it, the MongoClient will reuse any connection from it’s internal connection pool. What is a improvement that should be done in the future, is defining a static MongoClient over the application more details here

Now we will update location of a clientId i.e, inserting a new location along with it’s currenttimestamp inside ClientLocation for a clientId:

Nothing too difficult what the code does, it’s simply mapping LocationRequest to ClientLocation, here we are using the GeoJson, helper for mapping our coordinates from the request to GeoJson type defined the ClientLocation class. Lastly we are inserting a new document in ClientLocation collection using the async way.

You may ask, yourself where is validation? Right now, in this class there will be  no validation, this sole responsibility of this class is persisting data in mongo, I know the naming is not quite good, but I am open for any suggestions if there are any. The validations, will be put in the controller, at this time, and in the future we will create a class that will encapsulate all the logic of the validation and persistence calling, because we may use a different “protocol” other than HTTP, remember SignalR.

What remains in this class to be done is retrieving data and we’re done with a big part from our app.  Filtering data and retrieving it from a mongo collection is not a difficult job with new mongo driver, meaning that there are a number of ways of doing it but my preferred one is using a Build Filter as it shown in the next gist. More details can be found here

 

Breaking line by line we are creating a list of FilterDefinition of ClientLocation type. This list will be used in the end, for an, and operation, between all it’s elements. Next we include the filters, clientId, and predefined timeframe. Lastly we are mapping ClientLocation to a more friendly representation LocationResponse.

Now we have the the location manager class, next we need to have it included in our LocationController class and we can do some tests with postman/fiddler. Before that we need to make sure that mongo runs and the owin app is started.

If you use postman or fiddler you will first need to setup the Content-Type application/json in the header, or else you will get an 415 code (unsupported media type). As it is shown in the screenshot:

web-api-request-header

Next you will go on body and insert the Json for LocationRequest.  There is a long talk regarding date format, but for now, you can open the chrome console, or any javascript playground that is familiar and take the value of this script:

new Date().toJSON()

And after getting your output you can create a Json for LocationRequest, mine looks like this:

{
latitude:"0",
longitude:"0",
clientId:"ebecb4b0-a039-42e6-bb43-07ee1f9ef406",
currentTimeStamp: "2016-12-03T09:06:23.442Z"
}

As it can be seen a I used a guid generator to put something at clientId, so that we will not see a empty guid in the db.

Once this json is created select the raw option inside postman copy-paste, than, do send once the URL http://localhost:10280/locationserv/api/v1/client/location was defined along with post method. The postman should look something like this:

 

web-api-post-location

And once you hit send you should get an 200 code. And if you look in the db using robomongo or mongoshell, depending on your preference you should have an output like this I am using robomongo, but the output should be pretty much the same in the mongoshell,maybe the guids will be encoded in BinData:

{
    "_id" : NUUID("36a9c80f-1182-44e6-b2bc-34d6f7cb2f10"),
    "ClientId" : NUUID("ebecb4b0-a039-42e6-bb43-07ee1f9ef406"),
    "Coordinates" : {
        "type" : "Point",
        "coordinates" : [ 
            0.0, 
            0.0
        ]
    },
    "CurrentTimeStamp" : ISODate("2016-12-03T09:06:23.442Z")
}

Alternatively you could use the route:

http://localhost:10280/locationserv/api/v1/client/ebecb4b0-a039-42e6-bb43-07ee1f9ef406/location

And see all the locations of that client over time.

Wrap it up, now there is a functional web api, that receives, location from somewhere via http, and persist them in MongoDB, collection.  In the next post, we will continue refining our Web Api, with adding SignalR communication(no plumbing inside the owin app, that was done in the first part).

Posted in Fără categorie, MongoDB, Software development | Tagged , , , | Leave a comment

Web Api Service-Part 1

In the last post, we have created the first version of the UI. In this post, we are going to create the first part of the backend, that will be creating an owin self-host, define the setup, the routes and a test that everything should run.

  1. Prerequisites

Before getting your hands dirty there you will need to have installed the following:

  • Visual Studio Community edition
  • MongoDB(either latest version, or a version that supports WiredTiger as storage engine, as of version 3.2 wired tiger is the default storage engine previously it was the MMAP).
  • Robomongo(due to WiredTiger usage there should be installed the latest version which supports)
  • Fiddler/Postman which one you prefer for testing the Web Api, that we have created

2. Web API design

The responsibility of this API, is managing client locations. By that I mean the following:

  • clients sends it’s new location
  • client can get it’s previous sent locations in a dedicated time frame

Seems pretty simple, so we can have the following routes for the use case presented above.

  • Get method:api/v1/client/{clientId}/location
  • Post method:api/v1/client/{clientId}/location

Notice that I decided to use, a RESTfull approach, I am sure that it can be improved those two, but it is still a proof of concept.

3. Application

The, app, will be a self-hosted OWIN, app installed as windows service. I use this approach because right now, I think it is the fastest way of having something that we can touch. For the service part, I highly recommend the Topshelf library, which suits all the needs that you can possible have when developing this kind of app. As a way of communicating with MongoDB, I will use the latest driver that right now it is version 2.2 according to their site.

3.1  Initial setup

I will create a console app project in visual studio named AngularMap.LocationServiceApi. After that I will install the following packages

Next I create a Startup.cs class which will have the following:

There is nothing fancy, json serializer, cors allowed, basic SignalR setup(those values are on true for debugging purposes, on a production code I will put all on false for security reason), lastly I add a ExceptionFilter for logging the exception and this will keep me from adding additional try catch inside controller methods. Please note that is a rudimentary logging system, I encourage you to use a dedicated logging library along with a centralized logging system. Also this filter is not a silver bullet, for handling all the exception, on owin Web Api app, but it will suit the current needs, for any additional details follow this link.

 

After this I will create a separate class named LocationService that will start and stop, the owin app  along with the configuration:

Right now, all we need to do is calling this in Program.cs, and put a Console.ReadKey(), for making the app running in the foreground:

Until now, we have only the basic plumbing, what is missing, is a ping method for seeing that the web api  is configured correctly and works as expected. Next I will create a class LocationController, which will hold all the routes presented above, and we will use attribute routing so that you will see the route above the method of the controller.

Now if you start the console app, and then using Postman, and put the route: http://localhost:10280/locationserv/api/v1/location/ping . You should see a reponse like this one:

{"result":"pong"}

There is one modification that is needed and we can continue along. Including TopShelf for easily windows service install and also console app.  For that, we will need to change program.cs file.

This will give us the advantage of running as console app, and at the same time, we can open the exe file in the command prompt with install parameter and we can have it as a window service, far more simple than the sc.exe, if you ask me.

Right now we have a owin self-hosted app with web api routes defined. In the next post we will continue with MongoDB setup, along with finishing this web service.

 

Posted in Fără categorie, MongoDB, Software development | Tagged , , , | 1 Comment

Real-time google map dashboard

I am beginning my blog post series, with a mini-series, where I present and develop a proof-concept regarding a real-time map-dashboard available for web only, the idea is similar with waze map when you log in on a browser. It is not a production ready application, but a proof of concept.

Being a proof concept, application, I will evaluate the performance of MongoDB with new default storage engine WiredTiger. Also, on the front-end I will use angular 2, with it’s latest version(note:at the time at writing this article, angular 2 was still in RC category) and evaluate the ease of development. As on the communication/business logic  side I will use Asp.Net WebApi, along with SignalR and evaluate how these mature technologies behave on handling a lots of requests, by giving clients of this app two ways of communication:REST and SignalR.

To summarize the architecture will look like the following, starting from back to front:

  • Data layer:
    • MongoDB, with WiredTiger as storage engine. We will use a location collection where it will be stored the location history of the client;
  • Logic layer:
    • Asp.Web Api, REST approach for clients sending their location updates to the server;
    • SignalR, bidirectional communication:
      • alternative to the REST approach for sending their location updates to the server;
      • server updates to the clients, of different location updates, for SignalR clients.
  • UI layer:
    • Angular 2
    • Google map as map sdk.

The following diagram explains a little bit more the flow and how component interact with each other

ComponentDashboardArchitecture - New Page#

To be more precise, this app will have 2 kind of clients:

  • LocationProvider, which can be anything that can have a connection on the internet, and can communicate on HTTP protocol, or SignalR
  • UI-browser, where you see all the notification in that are made in the “field”.

The central part is the WebApi server, which will be an owin self hosted, that will send to MongoDB, the location history, and also sends location updates, of each individual LocationProvider, to UI-browser.

At this time, we observed that the real challenge of this app, if you are developing a production usage not a proof of concept like this one, is the performance in a highly usage environment with a lots of clients for each type. The performance part will have a dedicated post at the end of the series.

Next we will start the UI, with angular 2 and google maps. If you are the kind of guy that likes to start with back-end not front-end, you can jump straight to the back-end article and come back to UI.

 

Posted in Architecture | Tagged , , , | Leave a comment

Angular 2 Google Map app

Continuing the series, I am going to create a simple angular 2 app, that will “host” a google map. As environment I am using windows 10 and Visual Studio Code, for front-end. You can use other IDE, but right now, a lot of people recommend VSCode, for developing Angular 2 and, is very fast and is free of commercial use. Sure there is Visual Studio Community, but for the current requirements VSCode, does the job well. On the server-side I will use, Visual Studio.

  1. Pre-Requirements

As a pre-requirements for following the series, I assume you are working o a Windows machine, have the latest .NET version installed, Visual Studio, Visual Studio Code and NPM. Links that can be used for download:

 

2.Angular 2 setup

At the moment of the writing of the article there are 2 ways of setting up:

  • angular-cli: This will be the recommended way of preparing your angular app in near future, but at the time of writing the article, it is not a stable version.
  • manual way: Create the project in the manual way in VSCode, by following the guide from here  or the more faster way, download the project template from github .

I am not entering in more details of implementing this part, due to the fact that angular 2 is still not in release version and in the future some particular steps in the setup may change. Please follow the guide of starting an angular 2 app, before jumping to next step.

3.Google map

Including a map component to speak so, in your app, it will first resort to a map provider selection, the big players are: Google Maps, Bing Maps, Open Street Maps, Yahoo Maps . After selecting a map provider, you will have to include it in your app. All these providers have link, that gives you straight information about including it in you own, app, however due to the fact that angular 2 was the javascript framework, for client side, we need to take an modular/component approach. This means that, the map itself should be a reusable component which will expose some events and properties for manipulating it’s content. The SDK, presents them, but we need to make them in a more “angular 2 way” and at this point I don’t want to reinvent the wheel so I will use this library:https://github.com/SebastianM/angular2-google-maps  which uses Google Maps. It is a nice exercise of wrapping your own component by starting from this particular nice answer on SO, but is not the scope of this post.

Now let’s start including this library to our project. First we will go to package.json and add the map dependency:

 "angular2-google-maps": "0.15.0"

And this is how package.json looks right now.


{
  "name": "angular-google-maps",
  "version": "1.0.0",
  "scripts": {
    "start": "tsc && concurrently \"npm run tsc:w\" \"npm run lite\" ",
    "lite": "lite-server",
    "postinstall": "typings install",
    "tsc": "tsc",
    "tsc:w": "tsc -w",
    "typings": "typings"
  },
  "license": "ISC",
  "dependencies": {
    "@angular/common": "2.0.0",
    "@angular/compiler": "2.0.0",
    "@angular/core": "2.0.0",
    "@angular/forms": "2.0.0",
    "@angular/http": "2.0.0",
    "@angular/platform-browser": "2.0.0",
    "@angular/platform-browser-dynamic": "2.0.0",
    "@angular/router": "3.0.0",
    "@angular/upgrade": "2.0.0",
    "systemjs": "0.19.27",
    "core-js": "^2.4.0",
    "reflect-metadata": "^0.1.3",
    "rxjs": "5.0.0-beta.12",
    "zone.js": "^0.6.12",
    "angular2-in-memory-web-api": "0.0.20",
    "bootstrap": "^3.3.6",
    "angular2-google-maps": "0.15.0"
  },
  "devDependencies": {
    "concurrently": "^2.0.0",
    "lite-server": "^2.2.0",
    "typescript": "^1.8.10",
    "typings":"^1.0.4"
  }
}

After that we can either choose to install the whole packages or just the angular2-google-map library(that is a reccomended option if you follow the angular 2 guide for starting an angular app) by opening a command prompt where the package.json is on the the disk and issuing the following:

npm install angular2-google-maps

4.Wiring the component logic

Once the installation of the npm package is done we can start creating our map component that will use the angular2-google-maps directive.  For that we will use the angular 2 team recomandations in terms of project structure and in the app folder we will create a new folder called map and a folder shared(this will be for future use) as shown in the screenshot below. This screenshot shows how the project structure will look.

angular2-project-template

Inside the map folder we will add 3 new files:map.component.css, map.component.html. First we will start with the .css file, which sets the width and height of the map.

 

.sebm-google-map-container {
  height: 500px;
  width: 800px;
}

After that we will continue with the html file that will show the UI. Also we will add the posibility of adding some random pinpoints by clicking a button.




<div>
    <sebm-google-map [latitude]="lat" [longitude]="lng">
  <sebm-google-map-marker [latitude]="lat" [longitude]="lng">
  </sebm-google-map-marker>
  <sebm-google-map-marker *ngFor="let m of markers; let i=index " [latitude]="m.lat" [longitude]="m.lng" [title]="m.title" [label]="m.label" >
  
  </sebm-google-map-marker>
</sebm-google-map>
<button style="margin-top:20px" (click) ='addRandomGpsPoint()' >Add random point </button>
</div>



As shown in the html we are using *ngFor for adding markers on the map, also we are adding manually a marker to a default location that is set up inside the component logic/typescript file, we are using angular binding by adding square brackets to properties that are exposed by the map library that we are currently using. For someone who used angular 1 the square brackets might be looked like adding an ng-model directive backed, by a property inside the controller. Last, we are binding to a DOM event click with angular using round paranthesis, in angular 1 we would have used ng-click. Next we will do the interesting part, wiring the html with component logic using typescript. We will go into the map.component.ts and we will start defining the current module along with the imports:

import {Component,OnInit} from '@angular/core';
import {GeolocationService} from '../shared/geolocation.service';

Nothing fancy, we import the component and onInit, for handling the initialization part and also the directive from angular2-google-maps, also the geolocationservice that determines your current location more detail about the implementation here. Next we define the component syntax and module along with the whole flow:

@Component({
    selector:'map',
styleUrls:['app/map/map.component.css'],
templateUrl:'app/map/map.component.html',
providers:[GeolocationService]
})
export /**
 * MapComponent
 */
class MapComponent implements OnInit  {
    public lat:number=51.678418;
    public lng:number=7.809007;
    public markers:any[];
    constructor(public service:GeolocationService) {       
    }

    public ngOnInit():void{
        this.service.getLocation({enableHighAccuracy:false}).subscribe(a=>{
           console.log(a)
           this.lat=a.coords.latitude;
           this.lng=a.coords.longitude;
       },d=>{
           console.log(d);
       });
       this.markers=[];
    }

    public addRandomGpsPoint():void{
        console.log('adding some random');
        let lat=this.rand(this.lat-1,this.lat+1,0.01);
        let longitude=this.rand(this.lng-1,this.lng+1,0.01);
        this.markers.push({lat:lat,lng:longitude,title:'Title',label:'A'});

    }

    private rand(min:any,max:any,interval:any):any{
        if (typeof(interval)==='undefined') interval = 1;
    var r = Math.floor(Math.random()*(max-min+interval)/interval);
    return r*interval+min;
    }
}

Let’s break it down. In the first part @Component we define how this component will be used inside our app and also it’s providers along with html and css source. Next we expose some properties, lat,lng which have some defaults values but in the OnInit will be overridden with the current location taken from the browsers, and an array that will hold random points on the map in a predefined range of latitude on longitude see the methods rand and addRandomGpsPoint.

We use DI mechanism from angular, that will inject the GeolocationService class. That class has the @Injectable() annotation which tells angular that we can inject in any class we want only by specifying it in the constructor, from here we can go into nice testing patterns and usages, which seems far more simpler than angular 1 and far more nicer also.

Next for testing the app, we should do some updates. First we should include in our app component the newly created component. That means updating the view, more specifically @Component should have the template field updated with the mapcomponent selector. Next we should import the MapComponent module and exposed it as a directive, as you can see in the updated app.component.ts below.

import {Component} from '@angular/core';
import {MapComponent} from './map/map.component';

@Component({
    selector: 'ang-map',
    template: '


<h2>Angular Google Map </h2>



' +
    '


<map></map>



',

})
export /**
 * AppComponent
 */
    class AppComponent {
    constructor() {

    }
}

After this part we need to update the angular module along with the bootstrap with new settings.

import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';
import {HttpModule} from '@angular/http';
import {AppComponent} from './app.component';
import {GeolocationService} from './shared/geolocation.service';
import {MapComponent} from './map/map.component';
import {AgmCoreModule} from 'angular2-google-maps/core'

@NgModule({
 imports: [ BrowserModule,HttpModule,AgmCoreModule.forRoot()],
 providers: [GeolocationService ],
 declarations: [ AppComponent,MapComponent],
 bootstrap: [ AppComponent ]
})
export class AppModule { 
 
}

As it can be seen, NgModule is helpful, because we have declared everything that we will be using in this part of the application/module, and we are free of import in other part, as an exercise you can remove some of the imports in AppComponent and see what is happening. After the module is setup the bootstrap part(main.ts in our case) is pretty simple:

import {platformBrowserDynamic} from '@angular/platform-browser-dynamic';
import {AppModule} from './app.module';

platformBrowserDynamic().bootstrapModule(AppModule);

 

Once we have done this steps we can open a command prompt inside the root of the project and run npm start. And you should see something like this:

AngularMap

5.Conclusion

The development in angular 2, seems a pleasure, in contrast with angular 1, because of typescript full support, however, the development changes are pretty often and if I will have to choose angular 2 for a production use app, I will not use it due to those changes. If you worked in angular 1 and also tried angular 1 with typescript, than angular 2 will feel very natural for you. If you want to see the whole code you can donwload from here.

Posted in Angular | Leave a comment