Synchronize an AWS S3 Bucket in local file system

Dear Folks,
Today I want to synch a S3 Bucket on my laptop to move files (generated in last month by a crawler) in a new destionation.

The solution is so easy:

... in a bash ...

>export AWS_ACCESS_KEY_ID=....
>export AWS_SECRET_ACCESS_KEY=....
>aws s3 sync s3://test.lighthouse .

You have to refer to AWS CLI DOC for other details.

You can also do S3 bucket to S3 bucket, or local to S3 bucket sync.
Check out the documentation and other examples.

Regards.

Install Chrome stable version on CentOS 7.x

Hi Folks,
When you work with NODEJS it's tipical to add chrome to your software bag.

In some cases when create a unix image you works on CentOS, yum doesn't support immediatly the installation of a stasble version of chrome.

The configuration is easy.

Create a file called /etc/yum.repos.d/google-chrome.repo and add the following lines of code to it:

[google-chrome]
name=google-chrome 
baseurl=http://dl.google.com/linux/chrome/rpm/stable/$basearch enabled=1 
gpgcheck=1 
gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub

Check whether the latest version available from the Google’s own repository using following yum command:

yum info google-chrome-stable

Let’s install it using yum command as shown below, which will automatically install all needed dependencies:

yum install google-chrome-stable 

How to install Ubuntu updates via command line

Exceute following commands in the bash:

sudo apt-get update        # Fetches the list of available updates
sudo apt-get upgrade       # Strictly upgrades the current packages
sudo apt-get dist-upgrade  # Installs updates (new ones)
sudo apt-get autoremove    # Move packages that were automatically installed


Have a good time...

Documentation apt-get

Bye Folks

Protocol violations Pojects JDK 7 based and Public Nexus Indexes

In a very hot day of June a my collegue told me:

"Marco, I added some new dependecies to core project"

I thought "Here is it! At the first attempt won't comlie", I joked BUT

After a GIT pull the core project didn't compile.

The error was "failing in dependecy download -> procol violation"

Many public repositories changed the configuraion for HTTPS supporting only TLS 1.2.

We use:

NetBeans 8.1 powered by Oracle JDK 1.7 as Development Environment

It was claer the source of problem, the Oracle JDK doesn't support TLS 1.2 easily. My advice is to read this oracle blogs that introduce the topic.

The resolution was to change the jdk used to start netbeans. In our scenarion the change was from 1.7 to 1.8

In order to modify the jdk I accessed to directory:

<netbeans-inst-dir>/etc/

and edited the file netbeans.conf changing the proprty: netbeans_jdkhome

If you have problem on JDK configuration of your project, my advice is to add the old jsk as platform (not default) and configure it for every project in which the old JDK is mandatory.

Bye Folks,

Oracle JDK 9 and Eclipse Oxygen 4.7

Oh Shit!!!

WTF?

I downloaded JDK 9.. ehi guys we're talking about 350 MB installer ... ok not a problem, a 1 GB bandwidth doesn't suffer for few MBs.

The installation is the typical oracle raw package, jdk and jre (I want to analyze the server JRE option from Oracle, JRE and typical JDK tools to monitor and other features).

My idea was to analyze some feature of the last version of Eclipse IDE. Actually I develop with ECLIPSE neon III and OpenJDK 8.

I downloaded Eclipse Oxygen installer. I started the installer. In this case the idea behind the installer is a packages selector. Helpful? I don't now, I prefer zip archive (vintage lover? maybe!!)

The first installation is gone down.. Pay attention,. try to update installer before. The upgrade of installer fixes repositories references.

The second attempt wag good.

I launched eclipse and the result was ugly... NoClassFoundException ... I thoungth about new modules loader of java 9.

I found a nice article from eclipse foundation and the resolution of problem:

• add to eclipse.ini the parameter --add-modules=ALL-SYSTEM 

Finally, Eclipse IDE was opened in front of my eyes.

My evening was eneded with this error:

"Target is not a JDK root. System library was not found"

 Eclipse Oxygen doesn't support the JDK 9 installation for "JRE Enviromnets".

I'll search a solution to fix this.

In the past, I used NetBeans IDE to try the new features of JDK 8 for the same situtation. Eclipse didn't support the new java consutrct for lambda expression.

By allZ folks


EDIT, Floks follow this guide to start your Java9 training using Eclipse Oxygen.

My first experience with JUnit5 - Environment

I'm a TestNG lover. I think Junit is too limited for a complete customization of the test running phase.  I read many articles on the Junit 5 and its new features.

JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

• The JUnit Platform serves as a foundation for launching testing frameworks on the JVM. It also defines the TestEngine API for developing a testing framework that runs on the platform. Furthermore, the platform provides a Console Launcher to launch the platform from the command line and build plugins for Gradle and Maven as well as a JUnit 4 based Runner for running any TestEngine on the platform.

• JUnit Jupiter is the combination of the new programming model and extension model for writing tests and extensions in JUnit 5. The Jupiter sub-project provides a TestEngine for running Jupiter based tests on the platform.

• JUnit Vintage provides a TestEngine for running JUnit 3 and JUnit 4 based tests on the platform.

I appreciated the the attempt to improve the Junit framework significally, reducing the impact on AS-IS test suites based on Junit 3 or 4.

Some materils to approach new paradigm:

• https://www.slideshare.net/RafaelWinterhalter/migrating-to-junit-5

• https://www.slideshare.net/SpringCentral/testing-with-spring-43-junit-5-and-beyond

Clearilly:

• http://junit.org/junit5/docs/current/user-guide

I found interesting this github prject to start with  a valid configuration for MAVEN (gradle version is available):

• https://github.com/junit-team/junit5-samples/tree/r5.0.0-M4/junit5-maven-consumer

The junit5-maven-consumer project demonstrates how to execute tests based on JUnit 5 milestones using Maven. In addition, it showcases that existing JUnit 4 based tests can be executed in the same test suite as JUnit 5 based tests or any other tests supported on the JUnit Platform.

This sample project does not aim to demonstrate how to use the JUnit Jupiter APIs. For detailed information on the JUnit Jupiter programming and extension models.

Please note that this project uses the Maven Wrapper. Thus, to ensure that the correct version of Maven is used, invoke mvnw instead of mvn.

I think can be very helpful a simple JUnit Jupiter extension to integrate Mockito into JUnit Jupiter tests somewhat simpler.

The MockitoExtension showcases the TestInstancePostProcessor and ParameterResolver extension APIs of JUnit Jupiter by providing dependency injection support at the field level and at the method parameter level via Mockito 2.x's @Mock annotation. See here:

• https://github.com/junit-team/junit5-samples/tree/master/junit5-mockito-extension

Mockito is a mocking framework that tastes really good. It lets you write beautiful tests with a clean & simple API. Mockito doesn’t give you hangover because the tests are very readable and they produce clean verification errors.

If you have read the second link I reported (testing with spring) probably you can understand that mockito and some feature of spring ofr test are the same.

For y first test i used 'junit5 maven consumer', it implies the usage of surefire maven plugin.

I add two deails of my environment:

1. Junit 5 needs JDK8
2. Apache maven 3.3.9

This is the base of the my environment for unit tests.

Move from SVN local repository to GitLab.com

I had a great problem. I was working with Subversion repository, I'd prefer a gitflow policy to manage souce code repository.

My initial idea was to migrate to Atlassian BitBucket (http://www.bitbucket.org/). I had used in the past both version: free and paied (team group). It's a gret service expecially when BitBucket is integrated with Atlassian suite (JIRA, Confluence and so on, https://www.atlassian.com/software).

After a period of evaluation I discovered GitLab.com. This platform offers many versions of the solution, on cloud or on premise.

I adopted the free plan on cloud (https://about.gitlab.com/gitlab-com/):

Advantages

• Unlimited repositories
• Unlimited private collaborators
• 10GB disk space per project (including Git LFS and artifacts)
• Completely free, no credit card required
• Unlimited CI Runners to do parallel testing
• Alternate SSH port for git+ssh (443)

Features

• Unlimited public and private repos
• Create a new repo for even the smallest projects.

• Project importing
• Import existing projects from GitHub, BitBucket, Google Code, Fogbugz, or any git repo with a URL.

• Protected branches
• Control read/write permissions to specific branches.

• Wiki
• Keep your documentation within the project using GitLab’s built-in wiki system.

• Code Snippets
• Collect and share reusable code.

• Powerful APIs
• Control GitLab with a set of powerful APIs.

• Issue and MR Templates
• Create templates for issues and merge requests.

• Milestones
• Organize issues and merge requests into groups.

• Labels
• Categorize and track issues or merge requests based on descriptive titles.

• External Services
• Integrate with 24 external services (such as JIRA, Slack, Asana and more) to track progress across all the tools your team uses.

• Due dates
• Assign due dates to issues to make sure things get done on time.

• Powerful search
• Spend less time searching and more time building software.

I appreciated the WIKI, ISSUE, Slack Integration and the import repositories from BitBucket.

I've started to use the free plan. I create a group of project and I imported my projects from SVN.

In order to import repository it's easy.

Here there's the first problem with GitLab there isn't an available GUI to manage the resources easily on you computer, some operation are avaiable via Web Browser at gitlab.com. Clearilly, for git lovers this is an opportunity to use git bash.

Step to import projects follow:

1. Create a group via gitLab.com console (Optional)

2. Create a project under groups and save the reference to repository, something like this : https://marco.genova@gitlab.com/my-group/super-project.git
3. In local, checkout from svn the branch of the project to migrate, I've used TortoiseSVN (https://tortoisesvn.net/)

4. Access to git bash

5. Reach the directory target of the checout opertation

6. Lunched the command:
• git svn clone svn://myhost/svn/repos/project/Branches/the/last/branch

7. Launch the command, to expand buffer of HTTP connection
• git config http.postBuffer 209715200

8. Add origin to project via command:
• git remote add origin https://marco.genova@gitlab.com/my-group/super-project.git

9. Push on server:
• git push -u origin --all

10. Exit from git bash

After this step, I created development branch via Web Browser at gitLab.com.

A detail on SSH connection, I had all configuration for gitHub and I had a ssh key pair created yet so the operation was completed in 30 seconds (.. to Mars). The guide of gitLab is easy to follow:  https://docs.gitlab.com/ce/ssh/README.html#ssh

... For Raspberry PI lovers, youcan create a cluster of PIs with Swarn and deploy with Docker approach GitLab. I want to spend some time to implement this solution.

That's all Folks..

Please comment i f you have some tips or tricks on this approach.