Category: General

QA: An afternoon with the Rainforest QA test builder

By Emily Ellison | Leave a Comment on QA: An afternoon with the Rainforest QA test builder

Even in 2019, software testing is still a challenge for a lot of small companies. Testing is usually not prioritized amongst small teams. Small teams frequently lack a dedicated QA resource, this causes writing good tests to be a unique skill in itself. Because of this, teams will end up with maturing software products that have few or no tests. As development continues, the downside is that there is an increased potential for bugs to enter the product. So, how can small teams tackle this challenge? As the software development industry has evolved, the industry has developed a wide array of quality assurance (QA) tools and techniques. Broadly, these tools can be categorized into two buckets – manual (human) testing and automated testing.

Manual / human testing is essentially exactly what it sounds like. A human QA engineer manually executes a list of steps, evaluates the results, and decides if the tested software is passing. Manual testing is relatively easy to start because non-technical resources can develop and execute the tests. However, as the test suite grows, teams run into issues because they’re limited by how many QA resources they have. This leads to teams only running tests before certain deployments, causing them to miss bugs.

In contrast to manual testing, automated testing is typically entirely code based. A QA engineer writes tests in a general purpose programming language. This asserts that the tested software is still working as anticipated. Since the tests are executed by a computer, this approach does not suffer from the limitation highlighted above. The trade off is that, because the tests are written in a programming language, technical resources are required to develop the tests.

So, what if there was a hybrid approach that combined some aspects of each approach? Well, that’s why you’re here! Say hello to RainforestQA.

What is RainforestQA?

RainforestQA is a SaaS product that incorporates manual and automated software testing approaches. RainforestQA offers a free trial, followed by a pay-as-you-go billing model, where you only pay for the resources being used. RainforestQA tests are executed by an automated system, or human testers, depending on how the test is constructed.

What does an automated RainforestQA test look like?

The tests are composed of a series of steps, which describe actions or assertions, that the automated system must take. The steps can include, “load this page” or “scroll the page down,” while the assertions are things like, “see a button” or “confirm text on the page.” When any one of these steps / assertions fail, the entire test fails, which indicates that something is broken in the software being tested.

What’s the process of building these automated tests?

Building automated tests on the Rainforest QA differs depending whether it is written in Plain English or Beta Language. When constructed in English, the user is constructed to write their own question and answer for each step. If the test is written in Beta Language, an action can be selected from the sidebar on the right, followed by a target, which is also listed on the same sidebar. The types of actions and targets can be adjusted depending on the test and what is being assessed.

When composing certain tests in Beta, you will find that the same sequence of steps are needed. Instead of writing out every individual step, over and over, the “custom actions” feature can be used. This feature enables a series of steps to be grouped together, which saves a lot of time and energy. I found the custom actions feature exceptionally useful when a login was required at the beginning of the test. However, a flaw in this feature can appear if the actual custom actions, itself, is being tested. The test results for a custom action will not appear unless the results page is reloaded. While this is a very small detail, it was a fairly substantial inconvenience for me. The rest results appeared as though the custom action test was in progress for over an hour, when in actuality, the test results were returned within a few minutes, they just did not appear until the page was refreshed.

How does the back-and-forth work between users and Rainforest QA engineers?

When running a test, everything is sent through a real and active test team of almost 60,000 testers. The test team provides clear feedback in a timely manner. If the test is passed, it will appear in green (as pictured above.) If the test is failed, it will come back in red. If the “Go To Test” button is selected, the test feedback can be viewed. Specific comments and critique are given on the particular step that caused the test to fail. Additionally, all of the tests and results are automatically recorded and stored in a neat and orderly fashion.


What is the Difference Between Testing Languages?

As discussed, on the Rainforest QA, tests can be written in “Plain English” or “Beta Language.” Writing tests in plain english is faster and easier, but also much more expensive. For a test to be passed in “Plain English,” the tests have to be written and constructed in a very specific way. For example, if you wanted to test the login page while leaving the username or password blank, you cannot use the “type” action to exemplify that you are leaving it empty. With the Beta Language tests, you have to select a specific action from the bar on the right, followed by a target. The only choices are what is already listed. In Beta, you have the option to use custom actions, you can also make new targets, but only by labeling a pre-existing type of target. When conducting a test in beta language, screenshots are used to identify what should be seen/clicked on each page. The downside being, if there are three of the same buttons on one page, you cannot type in directions, nor can you describe which of the three identical buttons needs to be selected.


Conclusion

I haven’t plugged Rainforest into our development workflow, so I cannot speak on the integrations or reporting. However, I would recommend the Rainforest QA to anyone- regardless of their technical ability- that wants to run automated tests on a timely and inexpensive budget. Building tests on this QA very quick and straightforward. While you may find a few complications and specificities on each language, it typically would not take more than one revision to fix the issue.

TL ; DR

Likes:

  • Interface is easy to use
  • Variety of features available to test
  • Access to a test team that provides feedback quickly
  • Non-technical users can build tests and test the UI without writing code
  • Free trial and then pay as you go pricing
  • Custom actions

Dislikes:

  • Tests written in Plain English language ask for specific answers on tests which allows a huge margin for error including spelling, spacing, and plurals
  • Have to be written and constructed in a very specific way to be passed and you can’t use any screenshots to clarify directions
  • Screenshot feature for capturing targets do not always capture / appear
  • Cannot specify instructions on Beta language

Open Data: Which Greater Boston Regions Have Open Data Sets?

By Emily Ellison | Leave a Comment on Open Data: Which Greater Boston Regions Have Open Data Sets?

Open Data is defined as: “data that can be freely-used, shared, and built-on by anyone, anywhere, for any purpose” (Open Knowledge Foundation Blog.) Open data provides many benefits.

In a similar manner that it is essential to record a nation’s history, recording open data has comparable advantages. Keeping a running log of statistics and information can be used to analyze changes in patterns and sequences. With a measurable starting point, as well as updates, each community can stay informed and up to date about their surroundings. It is useful for the affected society not only to be aware of the changes in their government’s policies and implementations, but also the consequences. With mandatory government submissions and access to open data, local businesses have the ability to develop custom business plans tailored to their company’s surroundings.

Open data often includes demographic statistics in addition to employment information, salary, income, and spending. With open access, local engagement is welcomed and encouraged. Also, there is room for the public sector to make digital and technical transformations, implementing social progression and efficiency. Through this evolution, statistics on unemployment high school dropout rates as well as crime and violence can be targeted and countered.

To insure political justice, reporting open data is mandatory. This is essential for two reasons, it prevents the government from concealing certain statistics and information, and it is not gathered for a specific purpose. What this means is that the options for interpretation, analysis, and creativity are unlimited. People can use this data to make assessments and conclusions that the government may not have wanted to publicize. Additionally, this data can be used to measure and reinforce financial and economic status. From a technical standpoint, open data is very useful and endless in its opportunity for building.

Some examples of projects that have been produced with open data include: a school selection device, a flood print, online voting at events, home health and safety report, traffic and accident browser, damage from disasters assessment, a mobile voting ballot, etc.. The chart below provides the Greater Boston regions that have open data readily available. With this data, endless projects and tools could be designed, so, what will you build?


Interview: Chris’ Take on Tech in Real Estate

By Emily Ellison | Leave a Comment on Interview: Chris’ Take on Tech in Real Estate

How did you end up where you are today

I started in accounting, I worked in the CPA world with a real estate background. Over time, I drifted away from the CPA path. I knew it wasn’t for me. After CPA, I got an agent license and I thought my background provided a unique skill set to the industry. Thus, I went on my own and failed A LOT along the way, looking for a tech market fit in real estate.

How did you first incorporate tech in the real estate space?

With the help of engineers, my buddy and I built a pretty cool property search platform for buyers. There was a messaging feature, between agents and buyers, that we thought was state of the art at the time. After building a ton of legacy code that nobody wanted, we realized that we weren’t really solving a problem. The buyer search was occupied by major sites, like Zillow and Trulia, it didn’t make sense for us to compete there.

As you faced these challenges, what was your outlook?

I think many people perceive life is this perpetual climb to the top of a mountain and then you finally reach the top, raise your hands in the air, and scream “I made it!!” When in actuality, life is more like walking up 10 feet of the mountain, and then falling into a 30 foot drop off, meaning there isn’t a linear climb to the top. There are a ton of ups and downs. For me, it’s all about persistence and surrounding yourself with talented people, engineers and creatives especially.

So, what do you do now?

I currently own a real estate brokerage in South Boston. I work with a rockstar CTO and a talented digital team. We’ve built a proprietary database tool which helps us stand out from the rest, it basically tells us who is more likely to sell their home.

What inspired you to leverage tech/data for real estate?

I didn’t just want to be a traditional agent/broker and sell places. I felt I needed to add a ton of value in what I was doing and building. When I look back on what really started things for me, it was on the tech and data side. I’ve owned a place in Southie for some time now. I frequently get mailers from real estate agents, enticing me to sell. I actually received one yesterday! After collecting so many mailers, I thought to myself over the years, this can’t be the best way to reach owners. I mean I’m a broker, myself, receiving these mailers with no analytics on them. So, how many of these are actually read and converted into sale? A very small percent.

How do you identify a potential opportunity to use tech in a non-traditional situation?

I try to look at things from a high level and ask where the most value is added. In my industry, Redfin, the discount broker, has gained a lot of market share because buyers/sellers are getting smarter. They have begun to question the value of an agent. There are many agents covering too much territory in order to survive. When this happens, there is very little value add to the consumer. (i.e. if you have an out of town agent repping a buyer in South Boston, often times, there is very little value add for the buyer.) Whereas, if I was representing you as a buyer, I’d be able to give you neighborhood knowledge along with off market opportunities outside of MLS.

And for sellers, I can create an unmatched amount of bandwidth (targeting buyers) for your home because of our database technology.

What are some challenges you face building as a non-technical person?

Where do I start!? Haha. I actually want to write a book about this, maybe I’ll call it, The Technical Guide for Non-Tech Founders. I haven’t seen anything like it out there, have you?  Would you like to co-author with me? Making the right hiring decisions is a huge challenge, ALONG with building something a bunch of people actually need. Finding really great engineers to buy into your work can be tough. I’ve spent a lot of time in my early days on Elance (which is now UpWork) and you can get burned on there. I’ve always been bootstrapped, and when you make a mistake, it really really hurts.

Why do you think real estate is one of the last industries to adapt to new software and technologies? (calculations by hand and not through a database)

Great question! I think it is mostly because the business model has stood the test of time very well, even with the internet disruption in many other industries. 2005 and 2015, saw a wave of change. But, even with all the advancements on the home buying front, there is still a complicated and mostly analog process after the online search occurs. An end to end platform is coming and this will be interesting. Currently, there are some really talented people working on this.

Overall, how would you characterize the adoption of new technologies in the real estate space?

Real estate literally touches everything we do, because it defines our environment, our physical space. Think about this: of all the tech changes we have seen in our lifetime

ex. Amazon(goods), Facebook/Apple(communication), Google(information), Uber/Lyft(transport). Paypal/Venmo(Capital space.)

Real estate is larger than all of these categories combined. Real estate will evolve more quickly than people think. Venture Capital has poured a ton of money into the race, recently. The new wave startups are tackling a wide range of areas — building management, financing, co-working, appraisals, building amenities and empty retail space, even tech-enabled construction, management, and maintenance.

Do you think the future of real estate will evolve as they adapt to using new technology?

Yes, with the rise of technologies like autonomous vehicles, the drone, robotic delivery, decentralized workspaces, and other macro trends. These inventions are likely to lead to a complete transformation in how we utilize the spaces in which we work and live.

How to Lose Your Manager in 10 Days

By Emily Ellison | Leave a Comment on How to Lose Your Manager in 10 Days

Corporate cliches, like any cliche, should never be used. But you know what’s worse about corporate cliches, you have to hear them all the time, at a loud volume, from your manager that you don’t even like. Let’s call your manager Mike. Mike takes his job way too seriously, he always calls you out in front of your CEO when you don’t do something perfectly, and he has never taken one day of vacation in ten years. Mike shows up to every company gathering early, and complains that the pasta salad you brought isn’t chilled enough. You don’t even know who likes cold pasta anyway since you sure as hell didn’t make it yourself, and you’re just praying that Mike doesn’t say too much to your new boyfriend, if you even have a boyfriend by the end of the event. Well you know what’s the worst thing about Mike? I’ll give you a hint, it’s not his food preferences of lack of social skills … it’s his cliche sayings, his “corporate cliches” if you will.

Mike starts every call with a client by saying, “well at a high level..” he responds to every important question you have with, “let’s circle back” or “I’ll loop you in.” No, Mike, don’t do anything with me and round shapes. You are SO not hip. When mike disapproves of the work that he specifically requested from, he sends you “back to the drawing board” or “square one.” You want to know where you don’t want to go, Mike, there. But don’t worry, when you’re on ridiculous deadlines, he doesn’t tell you to go places, rather, he suggests you “hammer it out” or “move the needle.” Thank you for the encouragement, Mike… what makes these expressions so bad is that some of them really don’t make sense from a literal standpoint. Besides that, they are so overused. Just because your grandmother said them in her day, and hasn’t updated her jargon at the ripe age of 91, does not mean that it’s okay to take this terminology to the office place.

So, to help save what is left to be cultivated in the corporate world, we have made a game, Corporate Cliche Bingo. Every time someone says one one of these awful phrases, everyone else gets to fill in that square, until someone fills in 5 squares in a row, AKA BINGO. Maybe up the ante, a few drinks never hurt anyone. When Mike finally closes his mouth, or runs his wallet dry, you’ll have us to thank.

15 minutes with Puppeteer

By Ashish Datta | Leave a Comment on 15 minutes with Puppeteer

One of Setfive’s New Years Resolutions is to prioritize our internal marketing. In establishing online presence, an initial project included refreshing the @setfive Twitter following list. To do so, we built a list of target accounts that we wanted to follow and then started searching for tools to automate the following. After some research, it appeared the only existing tools were paid with weird, and “sketchy,” pricing models. So, we decided to look at using the Twitter API to implement this list ourselves.

As we started looking at the API, we learned you need to be approved by Twitter to use the API. In addition, you need to implement OAuth to get tokens for write actions on behalf of a user, like following an account. We were only planning to use this tool internally once, so we decided to avoid the API and just automate browser actions via Puppeteer. For the uninitiated, Puppeteer is a library that allows developers to programmatically control Google Chromium, which is Chrome’s open source cousin.

Puppeteer ships as a npm package, so getting started is really just a “npm install,” and you’re off to the races. The Puppeteer docs provide multiple examples, so, I was able to whip up what we needed in a handful of lines of code (see below). Overall, the experience was positive and I’d be happy to use Puppeteer again.

So why would Puppeteer be interesting to your business?

In 2019 APIs are popular, many business provide access to data and actions through programatic means. However, not all of them do and that’s where Puppeteer provides an advantage. For example, many legacy insurance companies only provide quotes after you fill out a web form, which normally, you’d have to complete manually. If you automated that process with Puppeteer, you’d be able to process quotes at a much faster rate. 24 hours a day, and give yourself a competitive advantage against your competition.