Shiny New Session Lifecycle Display

You might’ve already noticed, but your Solano CI report page may look a little different than you’re used to. We’re excited to release our newly redesigned Lifecycle display, giving you more insight into your build’s timing and progress.

Solano CI Lifecycle Header

This new lifecycle header splits your build into 4 logical sections:

  1. Startup – This phase includes the time it takes from receiving notification that you would like to start a build, to the time that Solano CI workers are assigned and your code changes have been pulled.
  2. Prepare – This phase includes any setup needed in order to successfully run your tests. This includes running setup and worker hooks, as well as initializing services you may have configured (postgres, mysql, elasticsearch, etc…)
  3. Run – This is where we run your tests. Nothing more complicated going on here.
  4. Finish – After your tests are completed, we run post build hooks, collect and upload logs, and complete any other teardown requested.

We’ve also made all of the timing data used to create this new heading available to users through our outgoing webhooks API.

Let us know what you think. This lifecycle update is a part of our ongoing focus on giving Solano CI users more visibility into every step of their CI build. Stay tuned for more releases in this vein.

Leave a comment

Managing AWS Credentials in the Cloud

Anyone who uses AWS and CI/CD long enough has invariably been told to store their AWS credentials in Environment Variables in order to use the AWS CLI to manage deployments or pull resources from S3. This is extremely unsafe for a variety of reasons.

AWS has been recommending cloud-platform providers to utilize the AssumeRole API to provide another layer of security and traceability to external AWS Credential access. As a proud APN DevOps Competency Partner, we’re excited to be the first cloud CI/CD provider to announce our adoption of AWS AssumeRole as the recommended & most secure way to use AWS Account Credentials with your Solano CI builds!

confused-deputy

Using AssumeRole w/ Solano CI

Follow the instructions here on setting up a cross-account AssumeRole policy in AWS. Make sure to follow the section that mentions “Allows IAM users from a 3rd party AWS account to access this account”.

You will be prompted for the External ID and AWS Account ID for the external account, you can find this information in your organization’s settings page.

Continue through the instructions, and copy the ARN for the new policy into the organization settings page and click “Save AWS AssumeRole ARN”. We will run a test to make sure the role has been set up correctly, and you will be presented with the following success message:

screen-shot-2016-12-07-at-5-44-05-pm

You can now use the AWS Code Deploy / Solano CI integration without the requirement of having to include Environment Variables with your build! You will also have access to the following temporary AssumeRole Environment Variables that will allow you to authenticate AWS requests using the new role:

AWS_ASSUME_ROLE_ACCESS_KEY_ID
AWS_ASSUME_ROLE_SECRET_ACCESS_KEY
AWS_ASSUME_ROLE_SESSION_TOKEN

Use these exactly as you would’ve used hard-coded environment variables, without the guilt of hard-coding your secure never-expiring credentials!

As always, please reach out to us via support@solanolabs.com if you have any additional questions!

 

Leave a comment

Announcing Solano CI Tutorial & New GitHub Integration

Solano Labs is excited to announce the release of our brand-new GitHub Integration!

Using the newly built GitHub Integration, we are able to provide a catered onboarding tutorial that shows you how CI/CD works within Solano CI – all using free open-source codebases. The tutorial walks users through:

  1. Forking one of our open source example repositories:
  2. Installing the Github Integration and granting access to your new fork
  3. Editing and automatically committing a solano.yml from Solano CI
  4. Watching the tests run and pass!

In addition to enabling this more robust onboarding flow, GitHub Integrations aim to give users more fine-grained control over the permissions granted to third-party applications. You can now give Solano CI access to your code on a repo-by-repo basis, eliminating the need for confusing bot accounts. The integration also gives the ability for Solano CI to have write access to your solano.yml, which will be used for smart  in-app editing of your Solano Configuration.

If you know of an Open Source project that has CI needs, and you’d like to see it featured as a part of our tutorial flow, please reach out to us at support@solanolabs.com.

This onboarding tutorial is available to all newly created trial accounts. Please give it a try and let us know what you think!

We’ll be giving the option for current GitHub OAuth Integration organizations to migration to the new GitHub Integration in the next few months – if you’d like to explore migrating sooner, please reach out to us at support@solanolabs.com.

Leave a comment

Introducing Solano Labs 30/30: 30% Faster Builds or 30% Lower Bill

Solano Labs is proud to announce our most ambitious campaign to date. Our 30/30 promotion offers you 30% faster builds or a 30% lower CI bill. No catch. It truly is that simple.

Our technology is the most scalable in the industry and we are happy to issue an open invitation for anyone to try it by providing an extraordinary benefit in the form of faster builds or lower cost. We are bringing the classic soda blind taste test to the enterprise software world, so try Solano CI today and experience our patented auto-parallelization and newly redesigned onboarding flow and decide for yourself. It is easier than ever to set-up a Solano CI account and get your build turnaround time down to #coffeetime–  the time it gets to get a cup of coffee.

Here’s how you can get in on the 30/30 action:
Spending over $1000 on CI per month?

  • Sign up for this special offer on this page.
  • Connect your repo and provide your current CI configuration.
  • Give us a few days to show you how much time or money you can save.
  • Pick Faster builds or Lower Cost.
  • Use the time or money you saved to throw a party for your team. Cheers!

Spending less? We got you covered too. Save 30% on your CI bill with Solano CI, connect a repo to get started.

Now relax and brew some coffee with the confidence that by the time your cup is ready your tests will be done and your team ready to continue working on core mission, not wrangling with test environments. Welcome to #coffeetime.

It’s a bold move, but we are confident in our industry-leading speed and customer support. For more information on Solano Labs 30/30 visit www.solanolabs.com/3030, visit us at AWS re:Invent (Booth #2320), or join the conversation on Twitter at #coffeetime.

1 Comment

Solano Platform Status Update- November 2016

First and foremost, we want to apologize for delays that some customers have recently experienced with Solano CI. We are in the process of addressing a chain of related issues that emerged over the last week. We’d like to explain what these issues are and how we’ve been addressing them.

The first issue emerged on Tuesday, Nov. 1, when Solano CI began to experience capacity constraints that limited the number of sessions we could support in our SaaS production environment. To the best of our understanding, this capacity issue was due to an unusually high demand for the AWS instance type that Solano CI commonly uses for its production workloads. For some time, Solano was unable to allocate new, healthy AWS instances with our preferred type and region.

In an attempt to mitigate this issue, the Solano team migrated capacity to different instance types that we determined as suitable backups when our preferred types are unavailable. These instance types required enough additional storage that using many of them bumped against an account-level volume limit size imposed by AWS; this killed some instances in use. Because of decreased supply (fewer instances available) and increased demand (restarting killed sessions), build queues backed up. We manually managed them during peak hours to limit slowness and worked with AWS to increase that storage limit, but it took multiple days to complete the process. With the higher limit, we are again able to spin up sufficient capacity for our full peak load.

Coupled with that, the high number of restarts affected other parts of the Solano system. Last week’s unusual load stressed Solano’s infrastructure, including the database, in unprecedented ways. Although most issues are resolved, we are working on one component that seems to be causing some customers’ builds to restart unexpectedly. This is still affecting queue throughput, and is now the critical priority for our engineering team.

Again, we apologize for the effect this has had. We understand how valuable fast builds are to your productivity, and are working to correct the problem as quickly as possible. We will continue to update this blog as more information is available. In the meantime, please don’t hesitate to contact support@solanolabs.com if you have any questions or concerns.

Sincerely,

The Solano Support Team

Leave a comment

Welcome to Solano- Test execution

Welcome to `Welcome to Solano!` In this blog series, we’ll examine a few key concepts and features of Solano Labs’ flagship product, Solano CI. The goal of this series is to help orient them while learning to use the system, while refreshing current users too; we hope you find them handy. Please let us know what you think or if you have any questions.

Cheers!

The key goal of any CI server is to execute automated tests written to evaluate whether software code functions correctly and thereby provide the development team with enough confidence to release that code to users and customers. While the notion of ‘functional correctness’ is a thorny subject, roughly speaking it can mean that the code causes its constituent app to behave in a way that conforms to the developers’ intentions as a discrete thing, and when interacting with the rest of the app’s codebase.

Automated testing, then, is crucial to the success of software development, since it provides feedback about the quality of the codebase and its behavior. And so Solano CI was created as a market-leading way of executing those tests in a fast and efficient manner. It does this by standardizing the method of test execution and providing easily understood reporting of those tests’ results. Below are brief explanations of the two primary means of controlling and implementing that execution: the `solano.yml’ file, and its parallel workers.

Solano will use the user-provided settings in a solano.yml file that is added to the code repo to be tested to determine which tests to run (or commands to execute). (See our docs on configuring with that file here.) The most typical way of selecting which tests you want to run is by test pattern, which filters down the whole list of test files to the desired subset using the syntax of the test file names. You can configure multiple patterns to run in a build.

The YAML file provides a centralized and version-controlled way of configuring the build environment, setup process, and tests run, as well as a source of truth for historical review of individual builds.

Once that set of tests is determined, Solano CI will use a scheduling algorithm driven by machine-learning that takes into account various metadata, like historical runtimes of each of the tests and their IO and CPU usage, to determine the most efficient order in which to execute the tests and the best way to subdivide this set of tests into smaller batches for optimal load balancing and compute utilization. Those batches are then automatically distributed to the parallel workers for execution. And as you continue to run builds in Solano over time, and add or remove tests, Solano will continue to automatically optimize to achieve the best runtimes for your builds.

As each batch is disseminated to the workers, Solano CI will then use whichever command you’ve configured to perform the tests. We recommend using the same command you use locally so that the results in Solano CI should be the same in both environments. If they’re not, we’ve prepared a Troubleshooting Guide that you should reference for suggested debugging steps.

Leave a comment

Solano CI integrates with Google Container Registry

screen-shot-2016-09-15-at-3-52-08-pm

Today, we are proud to announce our integration of Solano CI with Google Container Registry (GCR). With GCR and Solano CI, you can now reliably build, test, and deploy your Docker workflow without operating your own container repositories or scaling your infrastructure. GCR is a fully-managed, secure, and highly-available Docker container registry that makes it easy for developers to store, manage, and deploy Docker container images. In addition to the simplified workflow, you can configure policies to manage permissions and control access to your images using Google Identity and Access Management (IAM). There is no easier way to gain fine-grain permission control over your Docker images today.

We decided to create an example repository to demonstrate integrating GCR and Solano CI. Each commit to the repository will pull a container from GCR, trigger a build, and when all of the tests pass, the new Docker image will be pushed to GCR. Here’s the step-by-step process we used:

Set up necessary projects / permissions

Create a new project within Google Cloud Platform (GCP) for storing your container images.

You’ll need to create a service account, and download the JSON key file in order for Solano CI to authenticate with GCR. More information on creating a service account and downloading the .json key file are in Google’s support docs.

Once you have a new account, you’ll need to add the credentials to your Solano CI repository. Sign up for an account at our website and locate the repository you’re going to be using for your docker container. Download the Solano CLI and from within your repository, run:

solano suite

This will set up your Solano CI build environment.

Use the Secure Environment Variables UI or the Solano CLI to securely add the credentials:

# FORMAT: solano add:repo ENV_VAR_NAME ENV_VAR_VALUE
$ solano add:repo GCR_PROJECT_ID project_id_here
$ solano add:repo GCR_SERVICE_ACCOUNT_EMAIL service_account@email.here
$ solano add:repo GCR_SERVICE_ACCOUNT_JSON '{downloaded_json_blob_here: true}'

Push initial image to GCR

We’ll be pulling our initial image from GCR, so you’ll need to build and deploy your docker image using the Google SDK’s gcloud command.

Install the Google Cloud SDK

Follow instructions on pushing your docker image to GCR

$ sudo gcloud docker push $GCR_DOMAIN/$GCR_PROJECT_ID/$GCR_DOCKER_APP:$GCR_DOCKER_TAG

Set up Build & Deploy Scripts

The environment section of the solano.yml needs to be modified to use your container’s information.

solano.yml

environment:
 'DEPLOY_GCR': 'true'
 'GCR_DOCKER_APP': 'your_app_name_here'
 'GCR_DOCKER_USER': 'your_docker_user_here'
 'GCR_DOMAIN': 'us.gcr.io' (your region's registry url can be found here)
 'GCR_DOCKER_TAG': 'latest'

timeout_hook: 900

hooks:
 pre_setup: ./scripts/solano-gcr_build.sh
 post_build: ./scripts/solano-gcr_deploy.sh 

tests:
 - sudo docker run $GCR_DOCKER_USER/$GCR_DOCKER_APP:$GCR_DOCKER_TAG bash -c /run_tests.sh

Once your solano.yml looks correct, make sure to commit it to your repository.

Our example uses two scripts to build and deploy the GCR containers.  These scripts can be modified to fit your specific use case, but are set up now to use the environment variables supplied in the solano.yml

Start a Build

From within your repository simply use:

$ solano run

and a Solano CI build will initiate. It will run the build script which will attempt to authenticate using the credentials you securely stored in the first step, and if successful will pull down the container you pushed in the second step.

For the test phase, Solano will run the docker container that was pulled down, and execute the `run_tests.sh` script from within the container. If this `run_tests.sh` script exits successfully, Solano CI will start the deploy script.

screen-shot-2016-09-15-at-3-40-11-pm

The deploy script will tag the newly build container, and push it back to GCR for use in the future.

gcr

That’s all there is to it! As you can see, GCR is a powerful and robust tool for all your container storage needs!

For more information on deploying to Docker using Solano CI, visit our documentation or contact us, we will be happy to answer your questions :) 

Leave a comment

Secure Environment Variables UI

Background

While it has been possible to set secure environment variables from the Solano CLI for some time (see docs here), there has been no way to do it from the app or to see what values are currently set. We have added a new page to Organization settings that will allow Org admins to manage variables that are not set from a config file.

screen-shot-2016-09-21-at-5-26-35-pm

We are also adding a new tab to the report page that will list the environmental variables in order of precedence for a build. This page will display the list of Variables, their scope, and their value stared out. Org Admins will be able to toggle showing the actual values.

screen-shot-2016-09-21-at-5-27-10-pm

 

Have any questions or comments about this or any other Solano CI features – please don’t hesitate to contact us via email, twitter, or in the comments section below.

 

1 Comment

Easy Continuous Deployment You Can Trust ft. Apica

 webinar-easy-continuous-deployment-that-you-can-trust

Thank you to everyone who joined our webinar! In case you were unable to do so, then worry not. The full presentation, including every slide and resource, is available in this post.

In this presentation, Brian and Troy demonstrated a continuous deployment release process using GitHub, Solano CI, Apica, and AWS CodePipeline. The release process includes smoke, unit, integration and load tests to guarantee issue-free deployments, enabling a safe and easy push-button deploy process.

Brian and Troy also demonstrated how to:

  • Build your software release pipeline by connecting different steps into an automated workflow using AWS CodePipeline
  • Capture accurate and consistent performance data for each of your releases
  • Automatically stop a release when performance or errors don’t meet predefined criteria
  • Auto-parallelize your test runs with a Continuous Integration server using Solano CI

Check out the full presentation video here

Lastly, we would like to thank Apica for co-hosting this webinar, and AWS for their support. We <3 our partners!

Any questions/feedback? Let us know! We would love to have you present at our next webinar, stay tuned for details!

Leave a comment

Continuous Integration at BigCommerce

BigCommerce-logo-dark

This guest post was written by Deepa Padmanabhan, Lead Quality Engineer at BigCommerce, a large e-commerce platform enabling hundreds of businesses and processing over $9 Billion in sales. This is their experience with using Solano CI, as well as Solano Services to custom-tailor solutions to address their specific needs, and continue scaling seamlessly to deliver value to the customers that rely on them. Keep up with everything else BigCommerce engineering is working on by visiting their blog

BigCommerce is an e-commerce platform that powers hundreds of online stores across continents. Our hosted shopping cart service processes billions of dollars in total sales from multiple merchants worldwide. Customers rely on us to operate a professional website that generates online revenue for their business. Over hundreds of new features are added to the platform that enables clients to ‘Sell More’ successfully. It is imperative that such a powerful service be highly available, reliable and robust. Quality is critical at BigCommerce and we try relentlessly to deliver high quality products day in and day out for the customers who rely on us.

Our engineering philosophy advocates quality at every level starting from design to product release. With over 100 engineers working on our codebase and delivering product continuously we have a product lifecycle that requires us to follow rigorous testing process. Automating the tests contributes to a significant part of our quality process to ensure high quality and frequent deployment to customers. Every engineer takes immense pride in automating tests as new features are developed and automation team ensures validating every change using these tests. Here is the deployment flow we follow at BigCommerce to release high quality code to customers

 

Continuous Deployment Flow

continuous deployment flow

Automated tests have grown significantly and the need for regularly testing the quality of the builds and to catch issues instantly has led us to invest in a continuous integration tool. At BigCommerce we rely on Solano Labs for continuous integration and we are happy we went this route. We explored many other options including maintaining our own CI machines but Solano CI outweighed all of them for a number of reasons. There are several features that Solano CI provides that has greatly improved the engineering team’s efficiency.

 

Highlighting some of the notable features

Build Parallelization:

With developers merging code from different time zones, we had a large number of test builds queued up constantly. UI test suites took more than 5 hours to run causing significant delays in production release, less frequent test runs and accumulating breaking checkins. Solano CI’s build parallelization made this a breeze, where multiple test suites can be run simultaneously. Each build is further parallelized to run tests within a suite across multiple workers. This has significantly reduced the total time needed to run a suite. Our basic suite runs in less than 4 minutes and UI suite in 30 minutes thus significantly improving test feedback time. This has enabled us to implement feature branch monitoring thereby catching breaking changes even before they are merged.

 

Customization with yml:

Several customizations to fit our need have been made using the solano.yml config file. Ability to create stores per batch has made tests less flappy and avoids concurrency issues due to parallel test runs.  An internal implementation of test reruns, executes failed tests on the same Solano CI build and updates the final Solano CI output to reflect the rerun results. This has helped us in generating more robust results by eliminating flappers and avoiding manual validation of flappers. Solano Labs also went the extra mile to modify the Solano phpunit and suggested changes to custom test runner.

 

Profiles:

Grouping tests by environment and/or feature area is another important need for BigCommerce. We group tests by type (UI, API, HTTP), features (cart, checkout) and priority (release blocker, long running tests). Solano Build Profiles feature exactly translates to this need. It allows us to pass profile specific environment variables thereby targeting small test sets and executes only those. Solano API also provides a way to set environment variables per session on the fly thereby changing each test group and running different set of tests depending on the feature branch we are testing. This helps developers to narrow down and test only the product areas that are affected by their change and not wait for all the unrelated tests to run.

 

Solano API:

Solano Webhooks has helped us automatically kick off and monitor build status without manual intervention. Our deployment workflow has been highly customized to deploy every merge and kick off tests on Solano CI automatically. Integrating Solano CI with our internal deployment tool has helped us to automatically push to production on successful test runs or alert various channels calling for attention on failed runs. Using Solano API we have built an internal result monitoring tool that provides more granular analytics on test results.

 

Build Prioritization:

A new feature was rolled out just to address our immediate need where production release test builds were waiting in queue for long during busy days. This caused deployment delays and frustration among release team and customers. Profile prioritization enabled moving important test builds up the queue and to be picked up immediately. Production test runs are instant and don’t wait for long anymore. This has significantly improved release cycle time.

 

Customer Support:

Their unparalleled customer support makes it easier for us to address issues immediately. Session specific environment setup and build prioritization are some of the features they rolled out quickly that helped in accelerating our test execution time. Solano Labs has played a major role in helping us setup and rollout continuous integration within a short amount of time. Their continuous support has helped us go a long way in delivering high quality reliable product in a timely fashion. At the moment, every single merge to mainstream is monitored for quality and every single production deploy is guarded by regression suites run on Solano CI.

 

Contact us to learn how Solano CI and Solano Services can help address your organization’s unique needs too. To learn more about BigCommerce and their e-commerce platform’s latest, follow this link

 

 

 

 

 

Leave a comment