Announcing Marionette: It's like Selenium, but not

This is a project I've been working on for weeks and I'm very excited to finally have it to a point where I can announce it to the community.

Marionette was inspired by Puppeteer, and actually started as an attempt to port Puppeteer to Crystal. I learned after about a week of struggle that Node is much better adapted to an async workflow than Crystal. The lack of a good event system and promises made working with WebSockets in the way I needed quite difficult. I thought Marionette was doomed to failure until something changed in Crystal or the ecosystem, but then I learned about the actual Marionette protocol from Firefox.

The Marionette protocol is a TCP protocol built on top of WebDriver and is similar to what Selenium WebDriver uses. It allows near complete control of Firefox using TCP and JSON; including the ability to navigate, take screenshots, simulate pointer events, open and close dialogs, execute javascript, and more.

You can check out the project at https://github.com/watzon/marionette. The API is incomplete, but usable!

Subnet: Making working with IP addresses in Crystal easy

In Ruby there are a slew of different options for working with IP addresses, including the built in IPAddr class, but up till now Crystal has been lacking such functionality.

But no more! Subnet is a shard that makes working with IP addresses in Crystal not only possible, but extremely simple. It allows addresses to be used in Range literals, the creation of subnets and supernets, and many other things.

You can check out the readme documentation here or check out the api documentation here. And don't forget to give it a star if you like it!

Sorry for bombarding you with projects ;)

Announcing Arachnid: A powerful web scraper for Crystal

I love Crystal and have been using it for years, but one of the things I've felt like was missing for a long time was a good web scraper. So, because I definitely don't have enough personal projects, I decided to start building one myself.

Arachnid is based heavily off of the Spidr Ruby gem, with the main difference currently just being Crystal (and Spidr probably works better at the moment). The whole goal of Arachnid is to provide an easy to use DSL for scraping the web, whether you're looking for html pages, javascript, zip files, images, or anything else.

You can see some examples in the README and more will be coming very soon, as well as full documentation.

Announcing Strange: A more powerful logger

I know what you're thinking, "why another logger?!" I'll tell you why. Nothing out there currently has the features that I'm looking for in a logger. It seems like every logger in the Crystal ecosystem is super opinionated and lacks support for custom formatting and transports, so I created Strange.

Strange is a logger for those who want to do more with logs than just print them to the console. It supports all of the levels outlined in RFC5424, custom transports that allow you to define exactly where your logs go (whether that be the console, a file, or an external service), and custom formatters that let you format the output exactly how you want.

Strange is inspired by Winston, a powerful logger for NodeJS.

Please check it out and make sure to visit the issue tracker if you have any questions, a feature request, or just want to tell me I suck 😝

Cadmium: A pure Crystal NLP library

I've been working on a library to provide useful NLP (natural language processing) functionality in Crystal. The library is called Cadmium and is still in alpha with new features being added daily. Currently it provides functions such as tokenization, string distance, stemming, inflecting, n-grams, tf-idf (term frequency-inverse document frequency), and transliteration; with many more features coming in the near future.

Ngrok wrapper for Crystal

Ngrok is a well known http tunneling software for sharing local projects with the outside world. It can be an indispensable resource when developing websites, bots, etc.

I created ngrok.cr to make connecting ngrok to your Crystal projects just a little bit easier. It currently has examples using Kemal and Tourmaline, but it's fully possible to use it with Amber, Lucky Framework, or any other web-based framework.