with regard to Russian hackers

We have all heard of the boogeyman, right? He's for scaring kids to sleep. Russian hackers: they're for redirecting anger towards the unknown, and obfuscating the real issues with our 2-party system. Hopefully everyone has also heard of Pascal, and what is now known as Pascal's Wager. The question is, should one believe in God?

Basically, believing in God, per Pascal, is the smartest decision to make, because it avoids the only  possible 'infinite loss' which is that you don't believe in God yet he/she is in fact, real.

Basically, believing in God, per Pascal, is the smartest decision to make, because it avoids the only  possible 'infinite loss' which is that you don't believe in God yet he/she is in fact, real.

Great, let's try this with Russian hackers. It's a bit different, because our choice is one of priority, but it still looks similar:

fix hacking firstfix hacking 2nd
fix DNC fraud firstn/atakes care of itself
fix DNC fraud secondwe still have a problemn/a

Look, I get it. I don't like the idea of anyone hacking our anything, especially if it's a little bit true. But to quote GoTThe night is dark and full of terrors...

Seriously, there are a lot of problems with the world today. They all need to be addressed. I agree. It is a question of priority. Here are my premises:

1 - we as a nation are basically able to obsess over only 1 thing at a time

2 - Bernie would have won if the DNC primary had been fair

I don't think either of those things are even up for debate anymore. The ability to avoid Dr. Orange was baked right into the system. It was the DNC that failed us. Russian hackers were not and are not enough to sway the election, even if they do exist. If you grant the truth of those 2 things, THE ONLY DISCUSSION we should be having right now is how to either create a viable third party in this country (Bernie, I am looking at you) - or to somehow change the DNC for the better. If we do neither of those things, we are on track for season 2 of DOTUS™.

Coming to you in 2020.

But by all means, let's keep talking about Russia.

breaking up with MySQL

Breaking up is hard to do. I have been looking at this for over a month now and I'm seeing the writing on the wall.

We are still running 5.6, and we have run into a lot of issues getting the upgrade. Things like:

  1. Apparently 5.7 is so buggy that my PaaS provider suggests we don't do the upgrade.
  2. running 5.6 queries against 5.7 is a massive issue because the Group By syntax has changed, so just to get there every snippet of SQL that uses group by needs to be refactored.
  3. This is a tangent but I am very concerned about ownership of MySQL by Oracle. I am naturally cynical when it comes to private business, as I am a proponent of open-source software. I feel like businesses have become increasingly orthogonal to consumers, basically brushing them aside as they see fit, and over time it just devolves. I like to see a clear path stretching out into the sunset for the core products that I use, and for MySQL, I think those days are over.
  4. Everyone is jumping ship. I don't know anyone using MySQL anymore. Not that there is a significance there, necessarily, because I don't have that many developer friends, and I am not a blind follower, but it does perhaps suggest a simple truth - we have finally outgrown SQL and relational databases in general as the de facto answer for data storage. It is a more nuanced marketplace now.

So my angle will be to facilitate a move away from MySQL over the next months and years. Sounds like I have a lot of research to do. 

MariaDB?

PostGres?

NoSQL?

I am even looking at R right now to replace or augment our BI so things could get pretty interesting. We will see.

nosql-vs-sql-overview-1.png

There's a revolution going on in Business Intelligence part I

Just out of college, I got a job maintaining an Access database for a legal firm in Wisconsin that was vetting different types of records for legal discovery. As a 'temp worker' registered with the local staffing firm, I spent about 10 months writing queries and importing spreadsheets and scanning documents for a team of paralegals. With that experience, I was able to launch a successful software development career on my arrival to Los Angeles in 1999 and through to present day where I own my own 1-man code 'shop'. My main expertise is and has always been dealing with data, and managing the process from OLTP to OLAP and on to business intelligence, which is where a lot of progress is being made right now. Just in the last few years I have seen a preponderance of many new outfits in this space, and that is mostly a good thing but can be a bit wearying to vet so many similar products. I will be writing a few things around this topic so I am labeling this part I.

I am going to wait on doing a top list of anything, or start comparing products. But what I do want to do today is talk about Shiny. Here is a simple proof of concept that I created, where you can select a number of random 'seeds' and the application will automatically create a histogram of the normal distribution of those random numbers.

Now this might not look like much, but for me it is an absolute revolution. First of all, the speed - R is built for what you call 'medium data' analysis - which fits right between the 'small data' and the current buzz-word 'big data' (at least according to Hadley Wickham, chief scientist at RStudio). And even in the OLAP / medium data space, you run into speed issues. Constantly. Different platforms handle that differently, which I will talk about some other time, but right now let me be very clear - R-backed ShinyApp is way ahead as a service, because R is way ahead as a data crunching platform. The advantages vs. SQL are already very clear to me and the truth is that most businesses, large and small, will probably never get to a NEED for big data. And I can't possibly imagine a scenario where it would NOT benefit you to spend a long time playing with your data and creating models in R, even if you plan on going directly to Hadoop or BigTable or Redshift. It's just the right place to start.

And secondly, the workflow for generating this stuff is really revolutionary. It is certainly your best bet for having complete control over presentation at the same time of not having to render all your parameter selections with massive javascript blocks using something like AMCharts or Highcharts, both of which I have worked with extensively, both of which force you to manage a massive amount of custom javascript code inside your views to the point where it just feels messy almost immediately.

And finally, along with the speed of it, the reliability of the shiny app server is great. I have seen nothing to suggest it won't scale, and I think it will soon become a real competitor to the current leader in the white-labelled embeddable BI space, which IMHO is Mode Analytics.

Next up for this series will be to flesh out a bigger shiny app with some more meaningful data, which means I will have to navigate the date selection functionality as input, and use that to rewrite the output. I'm thinking of a day of the year slider, with maybe some heat maps so you can drag across a timeline and watch the daily micro-swings in denomination distribution which might be an aha for somebody if I hook it up to some sales data.

it's the privacy

Very often, I promise myself that I will write more. I am a writer, after all. That's my qualification. Sure, I've been writing software for web apps since the invention of the internet, but my degree is in Creative Writing. Go figure. So I go round in circles, because I want to create things, and I want to SHARE, but I can't just talk candidly about the people in my life, because that is a violation of privacy, and I am a pretty private person. But I also don't want to write things that no one will read. And as far as I can tell, unless I want to be a 'real' writer, and start my first novel, I am stuck being a blogger, and bloggers live and die by page views. Plus, my life revolves around a 4 year old and a 1 year old, and I don't feel right posting lots of photos and talking about them online. So it's quite a waste of time. I don't really even want particularly to be a successful blogger - I just want to feel like I am writing.

So what is there to talk about? Two things, really. The last few weeks were spent vetting new BI platforms, and although Tableau Online and Mode Analytics were both close, it became pretty clear that the only option for us is to continue with Periscope. They look the best, and they make my life the easiest. Sold. So here is a good example - my proof of concept was basically to recreate part of the main dashboard, and see how easy that is.

So, in Periscope, for the first chart the query is this:

select sum(amount) day_total, sum(f1) count_of_items, f3 merchant, ds
from temp_ps 
where p_type='Periscope Retailer 180'
and [f2=partner]
and ds > DATE_FORMAT(CURRENT_DATE - INTERVAL 31 DAY, '%Y-%m-%d')
group by ds, f3
order by ds, sum(amount) desc

Now, Tableau won't even let you give it a query, it likes to sit on a whole table and you drag fields around until you are happy. Not how I like to work. But Mode, on the other hand, looked pretty good, until you realize that:

  1. there is no Top X functionality;
  2. If you want to do that yourself you have to: write the query yourself, and handle the the settings in Python

So just as an example, the query, compared to above, for 'rolling your own' top x query - in this case, top 20:

select * from 
 (
( select * from
(SELECT (@rownum:=@rownum + 1) AS myrank,
merchant,
total
FROM ( SELECT merchant,
SUM(day_total) AS total
FROM {{@dash1}}
WHERE partner = "Facebook"
GROUP BY 1
) AS merchants,
(SELECT @rownum := 0) r
ORDER BY total DESC) z
where myrank <= 20 ) x
right join 
(
select sum(amount) day_total, sum(f1) count_of_items, f3 merchant, ds, f2 partner
from temp_ps 
where p_type='Periscope Retailer 180'
and ds > DATE_FORMAT(CURRENT_DATE - INTERVAL 31 DAY, '%Y-%m-%d')
group by f2, ds, f3
order by ds, sum(amount) desc
) y on x.merchant = y.merchant )

which as you can see is brutal, mostly because MySQL has no native ranking function, but to be fair that is before you even handle the presentation using python.

So it just quickly becomes a mess. I love the level of control, but I also need a simple workflow, and for Periscope you just click the damn checkbox. Yes please.

So it is nice to feel like I put us in the right direction. I hope that we can work with Periscope for a long time.

The only other piece of news is that I will be going to Austin at the end of the year to record an album with my buddy Shawn Pander as producer. I am super excited. Seriously this is going to be a pretty big deal. This is the song list (album working title: 'Nothing Can Keep Us'):

Go My Way
Roll Easy
Seeing Stars
You Got Me Running
Never Say Goodbye
A Little Dying
There's Nothing Beneath This
Dernier Coup

What Are We Doing

I was at a dance class with my young daughter. She is about to be 4 years old, and in about a month she will debut in her dancing career as a beam of sunshine. Cassie, the teacher, needed audio of each child saying their name for something she was putting together and each one of them needed to say ‘Hi, my name is…’ and finish with their name. So Cassie brought them all in to the office, which left me and the rest of the parents sitting by the wall as we do, waiting for what was barely 5 minutes.

And within 30 seconds, everyone had a phone in their hands, checking email or snapchat or Facebook or whatever it was. There was a mother close to me with a daughter in the class as well as a 1 year old son, and he and I caught each other’s gaze because there was no one else to make eye contact with. I smiled at him, and waved, it was a sweet moment but also sort of a sad one because he seemed perhaps a bit lonely right then.

A few minutes later, all the children filed back out of the office and the room livened up again.

Now, right here, I want to be clear that I am not trying to shame or blame anyone. I too am a parent, and even just a few minutes of respite is a non-trivial thing. The truth of it is that messages do need to be checked, and notes need to be taken, and text messages don’t write themselves. We can’t just stop doing it. The fucking light is blinking.

But know that when we all do that, all the time, there will certainly be more cases where a tiny little person is looking out at the world, trying to connect and finding nothing.

I think we can do better.

cross posted to kaxel.quora.com.
— https://kaxel.quora.com/What-Are-We-Doing

Holy Crap It's March - or How I Made My Lenovo Flex 4 a Ruby Slave

I just have not been writing. I can’t believe it’s March. These last few months have flown by. We had a flood in the basement, which is stressing me out a bit, and I had to postpone doing my taxes this year because Wells Fargo screwed up my paperwork, so that’s a bummer.

And I spent a few weeks prepping my laptop for a trip to San Francisco, and we had to cancel because everyone got sick. But we are heading over this weekend instead, so that’s good.

I guess I’ve just had a lot on my mind. My friend Mike passed away sort of suddenly from cancer, and that is just a tough pill to swallow. I feel super guilty because the last time I saw him he had his guitar out behind The Hotel Cafe, and was playing the blues, I was there for Sally Jaye’s release for Amarillo, and I just felt like he was going to be there forever. And now that was 10 years ago and that is the last memory I have of him. So sad.

And after looking at our finances for last year, we really needed to pull in the belt a bit. So when I was looking at getting a new laptop, for the first time ever, really, I seriously considered getting a Windows laptop instead of a MacBook because the price differential is pretty much out of control.

I settled on a Lenovo Flex 4 / Yoga, which ran me just under $600. The cheapest MacBook with the same specs would have been close to triple that. I just couldn’t justify that expense.

I set it up in about 2 days, and now it is running a Rails 3.2 stack and it is a fully functional work laptop. It’s amazing how far things have come.

So if you want to do your Rails Development on a laptop that is not a Mac, perhaps this list of steps can help you a bit.

1 - Windows 10, or nothing. You need Windows 10, because it is the Anniversary Upgrade for Win 10 that introduced something called WSL (https://blogs.msdn.microsoft.com/wsl/) ‘Windows Subsystem for Linux’, and that is what makes this all possible.

2 - You cannot access localhost from WSL, but the ip 127.0.0.1 works just fine. And Windows properly can access localhost inside WSL. So it goes like this:

Database - lives in windows, accessed by WSL server via 127.0.0.1

Browser, lives in Windows (I use Chrome), accesses WSL server via localhost

server, runs in Ubuntu

Editor, lives in Windows, accesses linux docs w symlink 

3 - Apps. You might be worried about what apps are available, but it seems like a decent environment at this point. I love my Mac apps - Fetch, Sequel Pro, Textmate - but there are perfectly decent replacements available. And for any old Textmate people, like me, you might want to take a look at Atom. The auto-complete works really well and reminds me of my old Visual Studio days inside the Microsoft machine.

I use - 

Cmder (http://cmder.net)

Atom (https://atom.io)

mysql command line client for Windows (https://dev.mysql.com/downloads/installer/)

NOTE for MySQL - the 32-bit installer does indeed install a 64-bit version if you want. That was not documented at all that I could see, and if you are looking for a 64-bit installer you are not going to find one.

Sqlectron (https://sqlectron.github.io)

and I haven’t checked yet but I’m sure I can find a decent ftp tool.

I used this walkthrough as a reference.

and the last caveats are that, for ubuntu, only Rails 3.2 and 4.3 are available; and MySQL is only supported up to 5.6 (at least as of a few weeks ago).

All in all, I am very happy with my experience with Windows 10. It has been pretty great so far.

The steps (roughly):

1 - install WSL

2 - test out bash command, set up aliases

3 - follow rvm instructions for installing on linux (use stable version - https://rvm.io/rvm/install)

4 - rvm install Ruby

5 - gem update --system && gem update

6 - sudo apt install git

7 - gem install bundler

8 - symlink linux docs folder back to windows documents -  (ln -s /mnt/c/YOURUSERACCOUNT/Documents ~/Documents)

9 - set up new key gen (I used the GitHub link as reference: (https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/)

10 -start trying bundle install, use apt-get as needed

for example: sudo apt-get install mysql-client-core-5.6, sudo apt-get install ruby-railties-3.2, sudo apt-get install ruby-dev, sudo apt-get install build-essential g++, sudo apt-get install libmysqlclient-dev will all be necessary

11 - gem install mysql2 -v '0.4.5'

12 - gem install activerecord-mysql2-adapter

and then the occasional sudo apt-get upgrade should keep you running.

good luck!

this is the best I have ever looked on TV

NOTE: This story was originally published on my old blog (from before I moved to Squarespace).

I would like to announce, to the media gods who are always keeping score, that this is the best I have ever looked on broadcast television. Remember me thusly when I am at the pearly gates.

screen grab from the CHEYENNE DVD (MTV 2006)

screen grab from the CHEYENNE DVD (MTV 2006)

I finally watched the dvd that I have had for 6 years now, and it is actually not a bad show. I remember how much empathy I had for her, she was such a sweet girl with so much going on and I can't imagine what it's like to have had so much success so early in her life. As turns out, I got on camera a little bit, which surprised me. My face was in the tag montage, which was cool. And the shot above is a screen capture of me walking on stage in Breckinridge, about to play keys and sing backup for CHEYENNE at our show there which was being filmed for an HD special. There are a few other shots of me in the earlier episodes, but this is far and away the biggest shot of me in terms of time and closeup level that has ever been broadcast on any channel of television. I have set my own bar in terms of media slam dunks.

The line-up was:
Cheyenne Kimball - acoustic guitar, lead vocals
Dave Krusen - drums
Joey Clement - bass, vocals
Krister Axel - keyboards, vocals
Travis Arnold - electric guitar