hayley 365

home

broken scrapers

18 Jan 2014

2014-01-18 23:00:17 -0600

So now I'm playing with snippets in Editorial as a way to start a Jekyll post.

What I'd ideally like to have is something where I give it a post title and then it gives me an appropriately named file in my _posts folder as well as starting me off with the proper yaml frontmatter. I'm not even sure if that's possible in Editorial. I know I could pull it off with pure Ruby though.

So on with it...

Today's official work was basically fixing the broken Nouvelle-Cal├ędonie scraper that I wrote yesterday (and had already assumed that it would break).

Later on, the Fiji scraper spit up all over itself trying to parse data for Ex-tropical cyclone June. I say, I do write the most fragile of scrapers.

Thanks to things like the fridge completely breaking down I haven't had a chance to work on what I'm supposed to be doing, which is explatory work to figure out what I'm supposed to use for authentication for my future web app.

See, I identified that it was the technical aspects of building a SaaS app that are throwing me for such a loop mentally and producing the most anxiety.

I'm absolutely expecting that the business side of things is going to take a lot of work. A lot of tweaking. A lot of time. A lot of pivoting. Just to find a single idea that is profitable. But for whatever reason, it's the technical stuff that seems so much more overwhelming.

The awesome thing about the technical stuff, is you basically solve it once. For instance, I'm not going to write an authentication system from scratch every single time. If I do happen to write one from scratch, I'll get to re-use it on all of my projects that need authentication.

And in some ways, maybe that's why I'm stalling. Maybe I'm secretly fearing what comes next far more than all of the technical stuff.

There is the fact though, that I did get like 50% of the way on one project on implementing authentication and Stripe payments. It never felt quite right.

Another problem is that Ruby is my preferred language and most of the super popular solutions and tutorials seemed to be geared towards Rails and/or only compatible with Rails.

Frankly, it also does not help that there is so much bloody fragmentation. Which of the thousand popular options should I go with?

And payments terrify me. What if I screw something up? How do I properly test without ya know, actually charging a card?

I know that things like Stripe have ways of testing but the whole thing is just big and scary.

Ugh. Just ugh.