Tuesday, March 14, 2017

Consumer Driven Testing with Pact & Spring Boot

Recently a colleague of mine stumbled across Pact.io,  Our current application had grown to over 50 services and we we're starting to have some integration test failures and a brittle dev / acceptance test environment. So we decided to have a look at ways to try help with this.

I started out by reading:
https://docs.pact.io/faq/convinceme.html

Then watching:
https://www.youtube.com/watch?v=-6x6XBDf9sQ&feature=youtu.be

Those 2 resources convinced me to give it a shot.

So I set out and created a quick set of Spring boot apps, the GitHub repo here, to test out the concepts and get everything working.

To highlight some important bits from the demo.

Consumer:
As Pact is a consumer driven test framework. This is where you define a unit test, that test mocks the http server response and you assert against that.
Once the test is successful it creates a pact json file in the /pacts directory.

So after the "mock" test is run and the pact file has been created. You need to include a maven plugin ...pact... that is then used to publish the content of the pacts/ folder to the pact broker... which is defined in the pom as below.

Producer:

This uses the JUnit integration from Pact.io to download the pacts from the broker and then run against an running service.

Since this already uses a @RunWith annotation, I could not use the spring boot runner. So to get around that as a before class step, I start the Spring boot application, the pacts then gets run against that running instance... and the boot application gets stopped again after the tests. Depending on your use case I guess it would also be an option to do this with @Before so you get a new service instance started before each pack, but that would slow down the execution tremendously.

The @State annotation, allows for clients to define a specific state, which the producer can the use to setup additional data / conditions required for the test to run.

Once the pacts have executed against the service there are reports generated in the target folder.




Setting up the Pact Broker

1. Grab the public images from Docker Hub.
docker pull dius/pact_broker
docker pull postgres

2. Then setup the Postgres DB
docker run --name pactbroker-db -e POSTGRES_PASSWORD=ThePostgresPassword -e POSTGRES_USER=admin -d postgres
docker run -it --link pactbroker-db:postgres --rm postgres psql -h postgres -U admin
CREATE USER pactbrokeruser WITH PASSWORD 'TheUserPassword';
CREATE DATABASE pactbroker WITH OWNER pactbrokeruser;
GRANT ALL PRIVILEGES ON DATABASE pactbroker TO pactbrokeruser;
3. Once the DB is up, run the actual Broker:
docker run --name pactbroker --link pactbroker-db:postgres -e PACT_BROKER_DATABASE_USERNAME=pactbrokeruser -e PACT_BROKER_DATABASE_PASSWORD=TheUserPassword -e PACT_BROKER_DATABASE_HOST=postgres -e PACT_BROKER_DATABASE_NAME=pactbroker -d -p 80:80 dius/pact_broker


Extra References:

https://docs.pact.io/documentation/
https://docs.pact.io/documentation/sharings_pacts.html
https://github.com/DiUS/pact-jvm
https://github.com/DiUS/pact-jvm/tree/master/pact-jvm-consumer-junit

Get the example project
https://github.com/bdupreez/pactdemo


Sunday, February 12, 2017

Quick Docker environment cleanup

Just a quick reference on my docker usage.

When working / playing with docker a lot, you quite often end up with a ton of containers and volumes laying around after they are needed...

Here are a couple tips I use to keep my environment clean.

On Windows 10
If you are using GitBash create a .bashrc and include the commands below:

alias docker-rm='docker rm $(docker ps -q -f status=exited)'
alias docker-rmi='docker rmi $(docker images -q -f dangling=true)'
alias docker-rmv='docker volume rm $(docker volume ls -q -f dangling=true)'

For MacOS X 
Add the following to your .bash_profile:

docker-rm(){
docker rm $(docker ps -q -f status=exited)
}
docker-rmv(){
docker volume rm $(docker volume ls -q -f dangling=true)
}
docker-rmi(){
docker rmi $(docker images -q -f dangling=true)
}
docker-update-all() {
docker images | awk '(NR>1) && ($2!~/none/) {print $1":"$2}'| xargs -L1 docker pull
}


Then simply run docker-rm, docker-rmi and docker-rmv to remove all the dangly bits :)

docker-update-all, is a handy way to get the latest images from docker hub.

Sunday, December 18, 2016

AWS Solutions Architect - Professional Certification

Let me start by saying, for this certification I studied and prepared more than any certification I have done before and I have done a number of certifications throughout my career; Microsoft, Sun, IBM, Oracle and now AWS. It does feel like there is a mountain of content to get through, and then ensuring you know it at the correct level of detail for the exam.

I booked the AWS Solutions Architect Professional exam 5 weeks after the associate level one and set a couple hours a day aside to study... During the last 2 weeks however, those couple hours became all my available hours every day, it was all I focused on, including 8+ hours a day on the weekends.

Main 4 resources I used:
  • YouTube
    • The re:Invent sessions videos are an essential resource
    • My major topics playlist: 


    • Watch these twice:

  • Linux Academy 
    • These guys really impressed me, the content in the pro level certificate was excellent, I found it had more depth than A Cloud Guru, well worth the $29 per month subscription.
  • Cloud Guru
    • Decent structured content that covers the highlights of all the topics defined in the exam blueprint.
    • The discussion forum is an excellent source of tips and discussions on the practice exam questions.
  • The official practice exam
    • This was a good eye opener, into how hard some of the questions can be. I do feel that some of the questions are badly articulated which makes them almost impossible to answer, however working through every question and researching the answers was invaluable. There were also a couple of those questions on the real exam... word for word.

Other Resources (Summary  / Study Notes):


Final thoughts:

All the guides and sites recommend reading a ton of white papers, but to be honest, I read only 4 or 5 of them..and I felt that the time spent was a waste. It seems all the content was in all the youtube vids and online courses already.

I did see a lot of people mention that they were pressed for time during the exam, but I found that I finished the 77 questions with a good hour to spare. To mention though: I did not go back and review any of my answers. I have always found in the past, that I rarely change my initial answer during the review phase, and I was reasonably confident that I had done enough.

End Result:

Overall Score: 83%

Topic Level Scoring:
1.0 High Availability and Business Continuity: 90%
2.0 Costing: 100%
3.0 Deployment Management: 85%
4.0 Network Design: 57%
5.0 Data Storage: 81%
6.0 Security: 78%
7.0 Scalability & Elasticity: 90%
8.0 Cloud Migration & Hybrid Architecture: 85%


Saturday, November 12, 2016

AWS Solutions Architect - Associate Certification

I have been messing around with AWS on little side projects and experiments for about the last year. I did find it quite a daunting experience as there is just so much to learn, do, try, break, struggle, deploy, re-learn and then eventually cheer when everything works.
So I decided to focus a little more and spend the time actually learning how to hopefully do all this properly. I have always found a good way to force myself into the theory and step back from diving directly into implementing solutions is certifications.

I booked the AWS Solutions Architect Associate exam for 4 weeks into the future and set a couple hours a day aside to study.

Main 2 resources I used:

  • A Cloud Guru
    • I did watched and worked through both the Associate Developer and Associate Architect courses. There is quite a bit of overlap, but still that just meant that I had covered some of the really important bits more than once.
    • I really enjoyed Ryan's courses, I did mostly watch on 1.3x speed as it does sometimes feel a little slow and I had set myself a 4 week dead line.
    • I would recommend them to anyone starting who's looking to learn about AWS, for practical real life use or certification.
    • The discussion section / forum is a great source of tips for the exams as well.
  • Cloud Academy
    • I only signed up for their 7 day free trial, so did not work through all of their labs. However they have a ton of quiz questions and explanations which is a great way to study / practice specifically for the certification.
    • Their summary video was also good and covered some content not covered in depth on a cloud guru.
    • They also had some quick easy reading blog posts highlighting important practical information when starting with AWS... listed in Other resources below.
Other Resources:

The main aws faqs to look at:

In the end I felt I had personally learnt a lot, not only on the exam topics but in general practical application as well. Main areas were especially with regards to VPCs, Networking and Security. It was well worth the intense 4 week study session.

End Result:

Overall Score: 90%
Topic Level Scoring:
1.0 Designing highly available, cost efficient, fault tolerant, scalable systems : 90%
2.0 Implementation/Deployment: 83%
3.0 Security: 90%
4.0 Troubleshooting: 100%











Next up.. AWS Solutions Architect - Professional

Saturday, October 8, 2016

TOGAF 9.1 Notes

I was considering doing the TOGAF 9.1 certification. I have gone over all the content, to at least have a decent idea, what TOGAF is and what it is trying to achieve.
However based on my current position, interests and trends I decided to rather focus my attention elsewhere for the moment.

So just to note down all the resources I found, used and other relevant resources links incase I do have the time or the need to come back to it.

Training course UDEMY (Scott Duffy):
https://www.udemy.com/togaf-enterprise-architect/
https://www.udemy.com/togaf-part2/
I would avoid the Exam Strategy one (https://www.udemy.com/study-togaf) as I found it a little bit of a waste, and he covers a lot of the content in the other 2 courses anyways.

Books / PDFs:
The Actual TOGAF specification (PDF)
TOGAF 9 Foundation Exam Study Guide
TOGAF 9.1 Quick Start Guide for Enterprise Architects (Not sure where I got a PDF from)
TOGAF Cheat Sheet from Scott Duffy's Udemy course.

Other Useful Resources:

Popular Posts

Followers