Ramblings on code, startups, and everything in between
One of Setfive’s New Years Resolutions is to prioritize our internal marketing. In establishing online presence, an initial project included refreshing the @setfive Twitter following list. To do so, we built a list of target accounts that we wanted to follow and then started searching for tools to automate the following. After some research, it appeared the only existing tools were paid with weird, and “sketchy,” pricing models. So, we decided to look at using the Twitter API to implement this list ourselves.
As we started looking at the API, we learned you need to be approved by Twitter to use the API. In addition, you need to implement OAuth to get tokens for write actions on behalf of a user, like following an account. We were only planning to use this tool internally once, so we decided to avoid the API and just automate browser actions via Puppeteer. For the uninitiated, Puppeteer is a library that allows developers to programmatically control Google Chromium, which is Chrome’s open source cousin.
Puppeteer ships as a npm package, so getting started is really just a “npm install,” and you’re off to the races. The Puppeteer docs provide multiple examples, so, I was able to whip up what we needed in a handful of lines of code (see below). Overall, the experience was positive and I’d be happy to use Puppeteer again.
So why would Puppeteer be interesting to your business?
In 2019 APIs are popular, many business provide access to data and actions through programatic means. However, not all of them do and that’s where Puppeteer provides an advantage. For example, many legacy insurance companies only provide quotes after you fill out a web form, which normally, you’d have to complete manually. If you automated that process with Puppeteer, you’d be able to process quotes at a much faster rate. 24 hours a day, and give yourself a competitive advantage against your competition.
Posted In: General
Posted In: General
Last week, a Symfony Live event was held in Paris. For those of you that are not familiar with Symfony, Symfony is a PHP framework which provides the tools to build an application. Symfony powers numerous popular applications such as the Drupal CMS, the leading open-source CMS, and serves billions of impressions per month.
What differentiates Symfony from other software creation templates is that Symfony is a community. The Symfony community emerged as an open source project where web application was developed to fit user’s needs. This community has since evolved as international conferences are being held to discuss the latest web application developments. The most recent Symfony conference, know as “Symfony Live,” was held in Paris. This conference included talks and workshops, as well as announcements on the latest Symfony updates and tools.
Some of the major updates announced at the conference were the new mailer and HTTPClient. We are excited about both of these components, as they will bring welcomed benefits in terms of not only developer experience (DX) but also efficiency.
Other talks and updates given at the conference included:
*most slides are written in French*
Posted In: Symfony
Okay, so it’s your first time making a website and you are not sure which CMS to use, you may not even know what “CMS” means. Don’t worry, you are not alone, because it’s my first time creating a website too! With zero previous website development, nor design experience, I found that looking up different systems and softwares can be overwhelming and confusing. I came across numerous compare-contrast charts and pro-con lists, but these responses meant virtually nothing to me since I did not know which functions and services I would need.
More specifically, after finding lists of which features were included on each CMS, I did not understand each features function nor its relative importance. So, I have constructed a little breakdown of my research, in attempts to assist those of you who are looking into making a website for the first time. Hopefully, by the end of this article, you will be able to better understand how to start thinking about creating a website, which factors differentiate each CSM, and what those differences will mean for you as a website creator.
So what is a “CMS?” CMS is an abbreviation for “Content Management System,” and a CMS does exactly that, it manages the creation of digital content. The two specific CMS’s I will be discussing are: WordPress and Squarespace. I selected these softwares because a majority of the research I found published these CMS’s as two of the most “universal” website creation platforms ie. can be used to create pretty much any type of website. Also, according to five different sources, Squarespace and WordPress are listed as two of the top three most frequently used website creation platforms. (builtWith.com, W3tech.com, SimilarTech, Google Trends, and websitesetup.org.) As the most popular CMS in the world, WordPress powers almost ⅓ of the world’s websites. With over 22 million active sites, WordPress has over 12.5 times more active sites than the next most popular CSM. Ranking a close third, is Squarespace. While considered universal- in the type of websites these softwares can make- WordPress and Squarespace act as systems that service solely for website creation.
With identical purposes, many of the functions on WordPress and Squarespace are uniform. Some of the major functions that are looked for in website creation include: content import/export, drag & drop, landing pages/web forms, online booking tools, online store builder, pre-built templates, SEO Management. Both WordPress and Squarespace (as well as many other CSM’s) possess the functions previously listed. In addition, WordPress has Auto Update and Survey Builder, where Squarespace does not. However, one feature on Squarespace that WordPress does not include is real time editing. On Squarespace you also have the ability for social media sharing, where on wordpress you can upload the links to social media sites, but you cannot upload the social media channel itself. Unless you are looking for one of those four differentiating functions between these two CMS’s, the bigger distinguishing factors between WordPress and Squarespace are the ease of initial use and the customization abilities.
The difficulty of learning how to use Squarespace can be understood through the analogy of learning how to read. There is a learning curve at first, but once you start to sound out letters and can read at a moderate level, that is pretty much it. Essentially, reading can be tricky at first, but once you are proficient in the skill, there is not more to learn or develop. You have the ability, and you can use it, just like you are right now. Reading is similar to Squarespace because, like piecing together letters, learning all the aspects and tools of the Squarespace software can be a bit tricky at first. The tools are broken into sections, and it is not clear how to start building. Before using Squarespace, it is recommending to watch some tutorials on how to use the platform.
However, a majority of Squarespace’s users state that it only takes a few days fully comprehend, and learn, the full layout. On the contrary, WordPress can be understood with the analogy of learning math. WordPress has straightforward tools and it is very basic and simple to start. Any beginner can instantly start a site build, however, it is nearly impossible to master all of the features and capabilities of WordPress. Specialists in building websites on WordPress claimed that they still do not know all the functions of this CMS even after using it for years. So, whether it be reading or math, Squarespace or WordPress, the initial phases of site building, and learning how to use the CMS, each vary by platform.
Whether you are creating a website for something visually appealing, like photography, or something more textual, like a law firm, my guess is that you want your website to look good. The ascetics of website are often considered the highest importance, and can often be a major fact in drawing clients towards your business. Somewhat obviously, the first thing that happens when we look at a website is that we see it. So visuals and layout are important.
With a high emphasis on the physical appearance on the sight, Squarespace can be tough to beat. Squarespace has- what are almost unanimously considered- the most aesthetically pleasing templates. The designs on Squarespace are also uniform and great quality. What this means is that if you switch templates within Squarespace, you can use the same designs and their quality will not change. WordPress, on the contrary, doesn’t ship with the most visually appealing templates. Despite the ascetics of the templates, the customizability between the two CSM’s is very different, which is where WordPress gains a lot of it’s users.
While the Squarespace templates may look nicer, there is a lot less that you can do to individualize your template. With these templates, what you see is what you get. Apart from adjusting font size and color, another website that uses the same Squarespace template will look almost exactly the same. Squarespace also does have the ability to cross share templates, so you cannot pick and choose which designs you like the most on each template. On the other hand, WordPress has the widest range of customizability. On WordPress you can customize every single feature and design, which allows you to create an entirely unique sight.
With WordPress you can build everything from scratch, where Squarespace templates are already formatted and pre-made. One visual feature that WordPress offers that some people prefer is the ability to upload a background. Again, another feature that can make your website more unique looking. Due to the major differences between ascetics and customization, Squarespace is a good fit for people that want to take a more simplistic approach to building their website. Since Squarespace requires no backend or coding experience, it is great for non-technical people. In opposition, those that are more tech-savvy, and interested in creating a complex site, may benefit more from WordPress.
The customization abilities of these sites not only differentiates them aesthetically and functionally, but also financially. With Squarespace offering minimal customization, the price is a relatively fixed rate. The initial rate is higher than WordPress because you are paying for not only a premade template, but also for Squarespace to host your website. There are many advantages to not hosting your own website. Squarespace is acknowledged for having a very responsive and supportive customer service team. Their team can be contacted via e-mail and Twitter 24/7 and via their site chat during standard working hours. Squarespace does not provide customer support through phone calls, but with an active team they are responsive and secure, and have great reviews.
If there is a bug or flaw with your site, the host is automatically responsible to fix the error. If you host your own site, you are responsible to repair any damages or issues with your site. Typically, this becomes expensive because you often need to hire someone directly to fix this. Contrarily, if you have backend coding experience, this can be a great way to save money, since you may be able solve the problem yourself. That being said, if there are any changes made to Squarespace ie. updates, changes, pricing, etc.. that is not something that a user can prevent since they are not hosting their own site. Without a support team, individuals hosting their own site also facilitate more security issues.
With your own hosting and customization options on WordPress, you also have more control over your pricing. WordPress itself is free, but hosting the site costs a small fee. This fee is less than the monthly fee for Squarespace. However, the more features and designs you add to your WordPress site, the more you pay. Thus, with increased customization options comes an increase in payment. The maximum that you can pay on Squarespace is lower than the maximum you could pay on WordPress, but, since you have to outsource your site to use Squarespace, you have no control over the rates the software changes its pricing. Squarespace’s current fees are listed below:
On Squarespace, your domain is only free for the first year. On wordpress, a domain name typically costs $14.99 / year, and web hosting normally costs $7.99 / month. Depending on your needs, your cost to start a WordPress website can range from $100 to even as high as $30,000 or more. I hope this article was helpful trying to understand some fundamental differences between Squarespace and WordPress. Another site with helpful insight one CSM’s is Capterra.
My conclusion is that Squarespace may be more suitable for entrepreneurs, small size businesses, startups or anyone who is less experienced in website building, or wants a more simplified site creation that they can outsource. Due to the ascetics of the Squarespace templates, this CSM can be extremely beneficial for visual people (photographers, artists, etc..) and ideal for portfolios and blogging. WordPress may be more fitting for freelancer, non-profit organization, even bloggers looking to craft a detailed and customized website for little to no initial costs associated. Specifically WordPress can be a great selection for this with technical or design experience that know how to host their own website. Happy choosing, and happy website creation!
Posted In: General
At the beginning of last year we tried something different and deployed an application on Amazon’s Elastic Container Service (ECS). This is in contrast to our normal approach of deploying applications directly on EC2 instances. So from a devops perspective using ECS involved two challenges, working with Docker containers and using ECS as a service. We’ll focus on ECS here since enough ink has been spilled about Docker.
For a bit of background, ECS is a managed services that allows you to run Docker containers on AWS cloud infrastructure (EC2s). Amazon extended the abstraction with “Fargate for ECS” which launches your Docker containers on AWS managed EC2s so you don’t have to manage or maintain any underlying hardware. With Fargate you define a “Task” consisting of a Docker image, # of vCPUs, and amount of RAM which AWS uses to launch a Docker container on a EC2 which you don’t have any access to. And then naturally if you need to provision additional capacity you can just tick the 1 to 2 and AWS will launch an additional container.
The app we deployed on ECS is one we inherited from another team. The app is a consumer facing Vert.x (Java) web app that provides a set of API endpoints for content focussed consumer sites. Before taking over the app we had learned that it had some “unique” (aka bugs) scaling requirements which was one of the motivators to use ECS. On the AWS side, our setup consisted of a Fargate ECS cluster connected to an application load balancer (ALB) which handled SSL termination and health checks for the ECS Tasks. In addition, we connected CircleCI for continuous integration and continuous deployment. We’re usingecs-deploy to handle CD on ECS. ecs-deploy handles creating a new ECS task definition, bringing up a new container with that definition, and cycling out the old container if everything went well. So a year in here are some takeaways of using Fargate ECS.
There’s a cloud devops mantra that you should treat your servers like a herd of cattle, not the family pets. The thinking being that, especially for horizontally scalable cloud servers, you want the servers to be easy to bring up and not “special” in any way. When using EC2s you can convince yourself that you’re adopting this philosophy but eventually something will leak through. Sure you use a configuration management tool but during an emergency someone will surely manually install something. And your developers aren’t supposed to use the local disk but at some point someone will rely on files in “/tmp” always being there.
In contrast, deploying on ECS reinforces thinking of individual servers as disposable because the state of your container is destroyed every time you launch a new task. So each time you deploy new code a new container will be launched without retaining any previous state. At the server level, this dynamic actually makes it impossible to make ad hoc server changes since it’s impossible to SSH to your ECS so any changes would have to present in your Dockerfile. Similarly at the app level since your disks don’t persist between deployments you’d quickly stop writing anything important just to disk.
When using Fargate on ECS the only way to access output from your container is through a CloudWatch log group through the CloudWatch UI. At first look this is great, you can view your logs right in the AWS console UI without having to SSH into any servers! But as time goes on you’ll start to miss being able to see and manipulate logs in a regular terminal.
The first stumbling block is actually the UI itself. It’s not uncommon for the UI to be a couple of minutes delayed which ends up being a significant pain point when “shit is broken”. Related to the delays, it seems like sometimes logs are available in the ECS Task view before CloudWatch which ends up being confusing to members of the team as they debugged issues.
Additionally, although the UI has search and filtering capabilities they’re fairly limited and difficult to use. Compounding this, there frustratingly isn’t an easy way to download the log files locally. This makes it difficult to use common Linux command line tools to parse and analyze logs which you’d normally be able to do. It is possible to export your CloudWatch logs to S3 via the console and then download them locally but the process involves a lot of clicks. You could automate this via the API but it feels like something you shouldn’t have to build since for example the load balancer automatically delivers logs into S3.
The ECS/container dream is that you’ll be able to run all of your apps on managed, abstract infrastructure where all you have to worry about is a Dockerfile. This might be true for some people but for typical organizations you’re probably going to need to run an EC2. The biggest pain points for us were running scheduled tasks (crontabs) and having the flexibility to run ad hoc commands inside AWS.
It is possible to run scheduled tasks on ECS but after experimenting with it we didn’t think it was a great fit. Instead, the approach we took was setting up Jenkins on an EC2 to run scheduled jobs which consisted of running some command in a Docker container. So ultimately our scheduled jobs shared Docker images with our ECS tasks since the images are hosted by AWS ECR. Because of this, the same CircleCI build process that updates the ECS task will also update the image that Jenkins runs so the presence of Jenkins is mostly transparent to developers.
Not having an “inside AWS” environment to run commands is one of the most limiting aspects of using ECS exclusively. At some point an engineer on every team is going to find themselves needing to run a database dump or analyze some log files both of which will simply be orders of magnitude faster if run within AWS vs. over the internet. We took a pretty typical approach here with an EC2 configured via Ansible in an autoscale group.
After a year with ECS+Fargate I think it’s definitely a good solution for a set of deployment scenarios. Specifically, if you’re dealing with running a dynamic set of web frontends that are easily containerized it’ll probably be a great fit. The task scaling is “one click” (or API call) as advertised and it feels much snappier than bringing up a whole EC2, even with a snapshotted AMI. One final dimension to evaluate is naturally cost. ECS is billed roughly at the same rate as a regular EC2 but if you’re leaving tasks underutilized you’ll be paying for capacity you aren’t using. As noted above, there are some operational pain points with ECS but on the whole I think it’s a good option to evaluate when using AWS.
Posted In: Amazon AWS