Web Monitoring Mechanism

In last years, it's normal to use javascript frameworks to test the javascript components in HTML pages or use framework as Selnium to implement regression test directly on a remote WebSite. I have a problem today, I want to avoid not-performed services so I need to examine a remote web site and understand the performances and the potentials not-blocking bugs on a retail portals (as a e-commerce). I need a Web Monitorng Mechanism.

These re some of the main goals of good service.  

My idea is to use PhantomJS to automate the test for some UI functionalities. PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.

PhantomJS is an optimal solution for:

• Headless Website Testing, run functional tests with frameworks such as Jasmine, QUnit, Mocha, Capybara, WebDriver, and many others. 
• Programmatically capture web contents: including SVG and Canvas. Create web site screenshots with thumbnail preview 
• Page Automation, Access and manipulate webpages with the standard DOM API, or with usual libraries like jQuery 
• Network Monitoring, Monitor page loading and export as standard HAR files. Automate performance analysis using YSlow and Jenkins

A good point of start is this: http://phantomjs.org/ :)

Page Automation and Network Monitoring are my features. PAY Attention for a good tool to monitor a infrastructure, my advice is to introduce nagios components https://www.nagios.org/ the scalability of this solution is the best BUT you need to study many aspects in order to create damage on you infrastructre.

My idea is quite simpler than nagios but the effectiviness is very good. I want to create test case with jenkins and phantomjs in order to test the UI functionalities.

I won't use Yslow. The last one  analyzes web pages and why they're slow based on Yahoo!'s rules for high performance web sites.

YSlow for PhantomJS also introduces new output formats for automated test frameworks: TAP (Test Anything Protocol) and Junit. With this new feature, YSlow can now be added to your continuous integration pipeline as a performance test suite, preventing performance regression to be pushed to production. (guide for this approach: http://yslow.org/phantomjs/)

The stack is quite different in my case.

My initial ide was to use the stack: PhantomJs, Jasmine (or karma) and Jenkins.

I've found this guide and I implemented the solution.

In my test Im' going to use the new UI of Jenknins, Blue Ocean (beta). The installation of jenkins is easy and totally web-based.

To start using Blue Ocean:

1. Login to your Jenkins server
2. Click Manage Jenkins in the sidebar then Manage Plugins
3. Choose the Available tab and use the search bar to find BlueOcean beta
4. Click the checkbox in the Install column
5. Click either Install without restart or Download now and install after restart
6. When installation is complete click the Use Blue Ocean button in the classic UI

Jenkins works perfectly yet with standard plugins.

PhantomJS itself is not a test framework, it is only used to launch the tests via a suitable test runner.

Jasmine is a Behavior Driven Development testing framework for JavaScript. It does not rely on browsers, DOM, or any JavaScript framework. Thus it's suited for websites, Node.js projects, or anywhere that JavaScript can run. Documentation & guides live here: http://jasmine.github.io For a quick start guide of Jasmine 2.0, see the beginning of http://jasmine.github.io/2.0/introduction.html

In order to integrate PhantomJS and Jasmine I tried to use Phantom-Jasmine. Phantom-Jasmine is a simple set of two scripts for running your Jasmine Tests via Phantom.js. The first script lib/console-runner.js is a plugin to Jasmine that outputs test results (with ansi color codes) via console.log (included with a script tag inside TestRunner.html). The second script lib/run_jasmine_test.coffee takes an html file as it's first and only argument and then executes any Jasmine tests that file loads. See  for more details https://github.com/jcarver989/phantom-jasmine (this is so and so... Too old).

The test implementetaion is easy so I tried to follow this guide: http://www.slideshare.net/WapAdmin/drupalcon-2013 . The limit in this case was to execute test directly in staging environment, pratically on a remote site.

With jasmine I have to modify my html files adding something like this:

<link rel="shortcut icon" type="image/png" href="jasmine/lib/jasmine-2.0.0/jasmine_favicon.png">
<link rel="stylesheet" type="text/css" href="jasmine/lib/jasmine-2.0.0/jasmine.css">

<script type="text/javascript" src="jasmine/lib/jasmine-2.0.0/jasmine.js"></script>
<script type="text/javascript" src="jasmine/lib/jasmine-2.0.0/jasmine-html.js"></script>
<script type="text/javascript" src="jasmine/lib/jasmine-2.0.0/boot.js"></script>

In the past I used jasmine or karma (for angularjs components) to test javascript components, in this case I used custom page to publish a component an d to test it directly in this custom page, the approach is the same but the difference is the browser used for the test session, in this case headless browser as PhantomJS.

I tried karma runner (https://karma-runner.github.io/1.0/index.html) and it works in the same way, I reused the jasmine tests using plugin for karma (https://github.com/karma-runner/karma-jasmine), I followed this guide to integrate in Jenkins: https://karma-runner.github.io/0.8/plus/Jenkins-CI.html

BUT

I want to monitor my remote application, it's not a only unit test. While I was reading phantomjs documention, I found CapserJS. CasperJS is a navigation scripting & testing utility for the PhantomJS (WebKit) and SlimerJS (Gecko) headless browsers, written in Javascript. (See http://casperjs.org/)

It's very easy to install, only few commands:

• npm install casperjs -g

Adding -g paramteers to add globally, for my purpose is a good solution.

The tests aren't compatible with jasmine format but I can reuse some part, my advice is to read this related information:

• http://docs.casperjs.org/en/latest/testing.html
• http://docs.casperjs.org/en/latest/modules/tester.html
• https://coderwall.com/p/ricgfq/end-to-end-testing-with-casperjs
• http://stackoverflow.com/questions/19946960/what-is-a-dubious-test-in-casperjs

My simple login tests is available here:  https://github.com/MarcoGenova/WebSiteMonitor

casper.test.begin('Testing Login on Backoffice', 4, function(test){

    casper.start('http://localhost:8080/myApp');

    casper.then(function(){

              test.assertUrlMatch(this.getCurrentUrl(),'http://localhost:8080/myApp/login.jsp');

              test.assertTitle('Login', 'The Web Site has correct title');

    });

    casper.wait(1500, function(){

        this.sendKeys('input[name="j_username"]', 'admin');

        this.sendKeys('input[name="j_password"]', 'adminpwd');

    });

    casper.then(function(){

       this.click(".isubmit");

    });

    casper.wait(3000, function(){

        test.assertHttpStatus(200);

        test.assertTitle('Logged Admin', 'The Admin is correct title');

    });

    casper.run(function(){

        test.done();

    })

});

PAY attention to this: ('Testing Login on Backoffice', 4

4 is the number of assertions aspected in this test suite, if the number is different for real asserts you'll experince a "doubitus test" and a test suite fail!!

In my scenario is important to publish the result of tests in xUnit way in order to publish automatically on JenkinsCI. CasperJS supports the log generation via parameter  --xunit=<filename> , example of execution is this:

>casperjs.exe test test-suites.js --xunit=log.xml

The last steps is to confgiure a new job on jenkins, the sub steps are these:

1. Configure the connection to GitHub, follow this guide: https://gist.github.com/misterbrownlee/3708738
2. Create a "Generic Build Job"
3. Add a nice name for the job
4. Add the reference to the project on GitHub (another can be used)
5. Enable the option "Delete workspace before build starts"
6. Add step in the build "Windows batch execute" (in my case winzoz 7)
7. Add the code "casperjs.exe test test-suites.js --xunit=xunit.xml"
8. Add actions after build:
1. Add JUNIT and fill the pattern field: "*.xml" (IT'S MANDATORY a relative directory), without this the nex step won't work
2. Publish xUnit Test after build
3. Add email notofication for every broken build

9. Save

Now you are ready to launch the build...

Sorry...  You need to configure mail server connection before launch... You can follow this guide: https://www.safaribooksonline.com/library/view/jenkins-the-definitive/9781449311155/ch04s08.html

The result is approved for my goals, probably I need to tune the configuration on e-mail notification and better discrimination of jobs in order to parallelize the process in the future, the pipelines'll help me for this purpose.

I hope to produce new elements on this topic in next months.

Give me some tips if you read this post :)

Bye

IKiosk application with Electron

Have you need to create your own kiosk.?!  Kiosk software is the system and user interface software designed for an interactive kiosk or Internet kiosk. Kiosk software locks down the application in order to protect the kiosk from users. Kiosk software may offer remote monitoring to manage multiple kiosks from another location. Email or text alerts may be automatically sent from the kiosk for daily activity reports or generated in response to problems detected by the software. Other features allow for remote updates of the kiosk's content and the ability to upload data such as kiosk usage statistics. Kiosk software is used to manage a touchscreen, allowing users to touch the monitor screen to make selections. A virtual keyboard eliminates the need for a computer keyboard.

In last years, I've seen the need to introduce some point in the landscape to help worker in own activities. Pay Attention, not all workers are engaged in IT technilogies.

It could be nice to place a kosk (as totem) to help a worker in a workerplace.

Many solutions exist but I wanto to create a very simple project based on Electron (http://electron.atom.io/)  and Windows Platform.

Electron is an open source library developed by GitHub for building cross-platform desktop applications with HTML, CSS, and JavaScript. Electron accomplishes this by combining Chromium and Node.js into a single runtime and apps can be packaged for Mac, Windows, and Linux.

Electron began in 2013 as the framework on which Atom, GitHub’s hackable text editor, would be built. The two were open sourced in the Spring of 2014.

It has since become a popular tool used by open source developers, startups, and established companies. See who is building on Electron.

Read on to learn more about the contributors and releases of Electron or get started building with Electron in the Quick Start Guide.

I've used electron for other applications and contexts. It can be very easy to build an application to interact with external context.

In order to develop a Windows Applciation we need to download Microsoft Visual  Studio [MS VS] 2015 (the community edition is perfect)... https://www.visualstudio.com/vs/community/

PAY Attention:  install Windowk SDK 8.1 too, this link can be helpful to resolve many problems to compile node modules and electro base code: https://www.robertkehoe.com/2015/03/fix-node-gyp-rebuild-error-on-windows/

We don't need MS VS to develop but to compile the native parts of node js. If you want you can use it to develop the code.

My advice start from this good guide from the web (the resolution of some issues are reported ): http://electron.atom.io/docs/development/build-instructions-windows/

Now, we can start with the application...

I want to create:

• The application must rendereize html5/css3/javascript content (Internal end External resources)
• The application must be ina fullscreen mode,
• The application must intercept all interputs to guide the behavior
• The application must log some information on file system or other appender (e.g. Remote logger )
• The application must start when the OS  starts

I've searached for a nice boilerplate code but I haven't found. So I started from a basic project.

I've created a Web Application (MEANJS stack based [https://meanjs.org/docs/0.4.x/#overview], out of scope for this post),  the Web Application read a QR code and check the code against a list of available code. This Web Application is hosted behind nginx (https://nginx.org/en/) on AWS (Amazon Web Services).

In the electron application...

I've integrated Browser Window:  https://github.com/electron/electron/blob/master/docs/api/browser-window.md so I can integrate Local and Remote HTTP resources, so I can access to remote pages via chrome pratically.

In order to cover the requirements of  the application in fullscreen mode., this documention on electron can be helpful: https://github.com/electron/electron/blob/master/docs/api/browser-window.md

Two apis can be helpful:

• To maximize your application window, you can use BrowserWindow.maximize().
• To run your application in full-screen, you can use BrowserWindow.setFullScreen(true).

I've added to BrowserWindow alwaysOnTop this lock the window. Pay Attention on this aspect. If an error occurs you can see it. My tip is to skip this configuration for the development and test phases.

NodeJS contains many module to interact woth OS level and intercept all types of event. The complex in some case is the recompilation od node to integrate new nodejs module in electron application, here the guide: http://electron.atom.io/docs/tutorial/using-native-node-modules/

Don't worry, this guide is based on the assumption that electron is compiled on the machine if you compile electron in theory you can integrate all modules supportec by embedded nodejs.

In order to cathc fails in the connection I've started some events from the webcontents loaded in the BrowserWindow so when an error occurs in remote page loading  then the application can start recovery operations.

A very nice aspect is to catch a string "from QRcode scanner" in emulated keyboard. NodeJs doesn't offer a very strong mechanism to capture the KeyEvent. When the system has to capture a data stream from a device (in keyboard emulation mode) , the developer has a list of possibilities:

1. Change the HTML source file to introduce a (interprocess communication mechanism) to pass information between nodejs main process and rederer process (https://github.com/electron/electron/blob/master/docs/api/ipc-main.md). The html page must implement the capturing phase of the data stream.
2.  Use globalShortcut. It's limited  but for more cases can be helpuful and not time consuming, https://github.com/electron/electron/blob/master/docs/api/global-shortcut.md

Actually I've uploaded a very light example of this scenario on GitHub:

https://github.com/MarcoGenova/mykiosk

If you want to help me to improve my example you can use the project. If you want to add some tips and tricks ... please ... Do it!!

I hope to add some functionalities to complete a minimal application.

Bye allz