Rebuild A Podcast by Tatsuhiko Miyagawa. Talking about Tech, Software Development and Gadgets.

00:00

miyagawa: はい、じゃあ今日のゲストはたこやき仮面こと、Aaron Patterson.

tenderlove: (laughs) Hello.

miyagawa: You just finished your talk.

tenderlove: Yes I did.

miyagawa: is this the first time you've delivered the whole talk in Japanese?

tenderlove: No, this is the fourth time.

miyagawa: Fourth.

tenderlove: Yeah, fourth. The first time was in Sapporo.

miyagawa: that was two years ago?

tenderlove: Was that two years ago? No.

miyagawa: I think it was.

tenderlove: Was it? Maybe it was. Yeah, two years ago when RubyKaigi took a break.

miyagawa: Yeah. That was a really great talk and your Japanese got so improved maybe?

tenderlove: Thank you.

miyagawa: I was so impressed.

tenderlove: Thank you. I've been practicing for long time. So the first time I did it in Sapporo it didn't go so well I thought. So, I couldn't make it the ... The first time I was supposed to give a 40 minute talk, but I accidentally, like I just went too fast it went it down to 20 minutes. It's way too fast. (laughs)

miyagawa: Yeah.

tenderlove: so I felt a bit better about the time this time I think.

miyagawa: yeah it was a 30 minute talk and you finished almost right on time, and still a slot for questions. Did you think those ダジャレ (dajare) and jokes by yourself?

tenderlove: Yes. (laughs)

miyagawa: That was really hilarious (laughs)

tenderlove: Yeah i had to keep thinking, I was like, I was planning the talk - so whenever I give a talk in English, I make dajare, but you can't translate those, so I have to come up with... so I have to come up with Dajare in Japanese! So I kept thinking and thinking and thinking and finally (laughs) finally I got something.

miyagawa: Yeah (laughs) Most of the dajares were really spot on.

tenderlove: Oh Thanks

miyagawa: "Rack は楽じゃない"

tenderlove: 楽じゃない (laughs)

miyagawa: ”高度なコード"

tenderlove: "高度なコード" Yeah (laughs) so I tweeted it and tried them out. After i tried them out, because like I don't know, I mean.. I don't know if it sits very well, because Japanese is my second language. So I don't know if it sits well. So after I tried them out, I tweeted like "高度なコード" and Nakamura-san was like, he was like "Oh, everybody says that" (laughs)

miyagawa: (laughs) The other one was not dajare. Maybe your talk will be published later on the site as an archive. One of that was showing a graph that displays the performance improvements. And you showed a graph of the y-axix with the weird scale, so that it is actually a tiny improvements but on the graph it was like a huge improvement (laughs)

tenderlove: but it is not! (laughs)

miyagawa: that's actually a common technique used in Japanese TV shows and media.

tenderlove: Ah yeah.

miyagawa: They make a huge, kind of confusing graph.

tenderlove: Oh yeah, it happens in the US media too, but I mean, I don't know - if you read it closely you can figure out, but it's not, I don't like when people do that.

miyagawa: (laughs) that was really funny.

tenderlove: but it's good to use this humor especially if you point it out.

miyagawa: Exactly.

03:42

miyagawa: So your talk was about improving the performance of Rails.

tenderlove: Yes.

miyagawa: Is Rail 4.2 currently in beta testing?

tenderlove: Yeah, it's currently in beta. I mean we're gonna release it as soon as possible. We're trying to work out any showstopping bugs that people have reported. We've got a few. The date's not completely set, but it should be, you know,...

miyagawa: Later this year?

tenderlove: Oh yeah definitely. Definitely this year. Just we don't know exactly the date.

miyagawa: OK. I wondered the other things you and other (core) members have done for Rails 4.2 is to speed up ActiveRecord. And I think that project was called AdquateRecord.

tenderlove: Yes.

miyagawa: Can you talk a little bit about it?

tenderlove: Sure. So I had an idea of speeding up ActiveRecord for a very long time. Maybe I've been working on that project for maybe three years or so. And what it is is basically like, all it is is just, every time you generate a query,

miyagawa: SQL query.

tenderlove: Yeah SQL query. What ActiveRecord would do is create an AST representing the SQL query then it would compile the SQL query into the actual string. Then it would send the query to the database. And the problem is if you look at the queries in your system, many times (they're) very similar. Because you're doing the same query over and over again, but we have to do that same process over and over again - create the tree, then compile it to the SQL statement. So what this project was is basically to make it so that we only do that compilation once. So you compile it down to the string and we cache that. And every other time, every subsequent time we just reuse that thing so we don't have to do the compilation step anymore. So that's what the project was.

miyagawa: How's the progress? Is it already merged into 4.2?

tenderlove: Oh yeah. It's actually been in master for a while.

miyagawa: OK. When you cache those queries, what's the cache key?

tenderlove: Um, that's a good question. So the cache key depends - well, the cache key is the table along with, er the model along with the parameters for the query. So like, if you do User.find_by_name then the cache key will be User and name. And if you do like User.find_by_name_and_whatever, it will be a combination of those things. That's what we use for the keys.

miyagawa: So if you have the same method calls on, for example User model, in different places in a controller, they will share just one cache.

tenderlove: So if they're always doing find_by_name it will be the same cache, yeah.

miyagawa: But if you chain those Active(Record) Relation,...

tenderlove: Yeah, so that's one limitation. One limitation of the cache is if you do something like User.where(:name =>).where(:id => ...).where and if you keep chain stuff onto that, it's much more difficult to cache, we don't have a cache for that in place. That's - actually one of the plans I have for the future is it's possible to cache those types of queries. And if you look at AdequateRecord's implementation, it's actually just using those .where.where.where chains - it's using those and caching those, so it's possible to cache those except that right now you have to have a specific block, so you say like "OK, cache this thing." So we do that under the hood in ActiveRecord but we want to come up with the - a future plan is to come up with something more automatic, right. So you can continue to use just the same code and it will get faster.

miyagawa: Oh cool. How are the actual improvements? Did you measure it?

tenderlove: Yeah, so it depends, like ... when you just measure something very simple like micro benchmarks doing find_by_name or find_by_whatever, we'll see something like 2 times or more increase. It will actually go above 2 times, but we're seeing an average 2 times faster. And if you look at - it's hard to gauge that impact on an overall system, because we don't know what queries you're doing in your system, that's hard to say, but I did measure it against just a scaffold base, you know, scaffold Rails application and we saw on the scaffold, maybe 10 to 15% throughput improvements. So, I mean, 10% better on scaffolding seems OK.

miyagawa: Yeah, seems good. Nobody will complain about the performance increase.

tenderlove: No, as Matz were saying (laughs)

miyagawa: if they get that for free (laughs)

tenderlove: Yeah exactly, yes. (laughs)

miyagawa: "I have this problem with Rails - it's 20% faster"

tenderlove: "Yes it's too fast!" (laughs)

miyagawa: (laughs)

tenderlove: "What do we do to slow it down?" (laughs)

09:29

miyagawa: Yeah. It's interesting - the name AdequateRecord. When we see the gems and extensions to Rails, I think it's been something like Active-something, ActiveQueue, ActiveModel... And then recently with Rails 4, DHH started some other extensions which is enabled by default, like TurboLinks and StrongParameter.

tenderlove: Ah yeah...

miyagawa: It's like really shiny, strong, turbo, fast. And you're like "Adequate" Record. It's really a humble and modest expression. What's the ...

tenderlove: Yeah I don't know. I'm just tired of Turbo!, Active! ... all that stuff. I was like C'mon, it's just code. (laughs) Seriously (laughs)

miyagawa: What about "Lightning Record".

tenderlove: Yes, "Lightning Record!". (laughs) I wanted to name it - the reason I named it that is because, I think everybody sees those names, and they after a while, I think, it's just a name (laughs) So I wanted to use that name because now we all know, that it's just a name (laughs)

miyagawa: Yeah (laughs) Adequate Record.

10:54

miyagawa: What's next for ActiveRecord?

tenderlove: Well doing the caching I was talking about there. Other things I wanna do are like, the stuff I'm working on at RedHat, we have some extensions to ActiveRecord that I wanna start pushing upstream, but to be honest most of them are Postgres specific. I don't know, things like, it's weird - we don't have - I didn't realize this but apparently Rails by default uses 32 bit integers on Postgres.

miyagawa: OK, not 64 bit.

tenderlove: Yeah, you have to specifically say like "I want a 64 bit integer column" and I don't know why - well I do know why now, we started out with 32 bit, and the problem is when we dump the schema, when you look at the schema.rb we use integers for the primary key column, but if i remember correctly, in schema.rb we don't say the type. If it's integer we won't say INT, we say this is the primary key. So we have to start dumping that information, otherwise we'll mess up when people upgrade.

miyagawa: Is that to translate to bigint on the database level.

tenderlove: Yes, exactly.

miyagawa: I remember that. I wanted to do bigint for MySQL driver, and by default it was 32 bit. So I had to, basically, at that time - maybe it was 3 years ago so it may have been improved, but at that time, what I needed to do was, start with integer and right immediately after, I issue an alter table with the database migration.

tenderlove: change the 32 bit to 64 big int?

miyagawa: Yeah.

tenderlove: Yeah. So I think what we'll need to do is we need to start - I mean the first step is we need dumping all of the information in the schema.rb, including the primary key type. Once we do that, then we can have people use bigints and then it will upgrade correctly.

miyagawa: Yeah, a few weeks ago I had Matsuda-san on the show, and the thing we talked about is (that) ActiveRecord is in Rails core, but that's one of the areas that clearly lacks maintainers on for example MySQL, Postgres like you mentioned. Now that you joined RedHat and probably do some Rails work there, do we expect a little improvements on the - enhancing ActiveRecord much better?

tenderlove: Yeah i think so. I mean as we get - it just depends on what I can pull out of the application that we have at work, but yeah i mean I keep working on it, I've been paring with the woman named Eileen, she's eileencodes on twitter. We've been working together on improvement to internals. Also a guy named Sean Griffin, he's been working on a lot of stuff too. So I mean, actually one interesting thing that he did was, he's been changing the typecasting code internally, so we can support arbitrary types lot more easily. It's easier to support things like Postgres' JSON types.

miyagawa: Array types, HStore.

tenderlove: Yeah, exactly. Any of those. It's easy to add any of those custom types now.

miyagawa: Cool.

tenderlove: I think that's a pretty big improvement. Although I have to admit most of the features, most of the improvements that I've seen are from the Postgres side. I haven't seen much going on on the MySQL side.

miyagawa: OK.

tenderlove: I think it will be cool if we had better GIS support. That would be cool.

miyagawa: Geo information?

tenderlove: Yeah.

miyagawa: That would be huge.

tenderlove: I think that will be really interesting. Just, we are not doing any geo stuff at work, so...

miyagawa: Is that the area the developers can contribute a lot if they want to?

tenderlove: Oh yeah, absolutely. Especially with this new typecasting system that's in there, it should be really easy to start adding geo types to it.

miyagawa: Great.

15:23

miyagawa: So the other thing you mentioned was the Rack. Rack は楽じゃない.

tenderlove: (laughs)

miyagawa: You just announced - it was actually announced on the mailing list so it's (been) public, but Rack, the development is going to fade away. Its not going away, the actual code will be there for a long time, but Rack 2.0 will not happen as Rack it is today. There's gonna be a lot of experiments on what Rack 2.0 is gonna to be like, and you started this experimental project for that.

tenderlove: Yes. So I started the project called the_metal, which is an experimental version of Rack 2.0. So basically what happened was is, the most of the Rack team was like, they moved on to other projects. Improving the current API is just too hard. I mean the main problem we have is the env hash - Rack env hash - if you look at for example, hijack API - the hijack API was like "OK we're just gonna put a special key inside the hash" and it's just - it's not very extendable.

miyagawa: Right.

tenderlove: So what I wanted to do is basically switch it to more object typed. I think what we should be doing is we should be taking advantage of Ruby's OO. Ruby is so strong with OO, we should be leveraging that for our web application. My idea with themetal is basically start using more objects, and eventually I want themetal to just die, I want that to go away, I want it to become Rack 2.0. So Rack will just continue. I mean for now, I'm taking over maintenance of Rack 1.x, we'll keep working on that, fixing bugs, releasing it and stuff. But I wanna be the one driving 2.0. So I mean ... Rack は楽じゃないといったけど、未来はあると思う.

miyagawa: 未来はある.

tenderlove: Yeah.

miyagawa: That's good. rack-hijack, I think, was to jailbreak out of the Rack environment, and do whatever you want with the server adapter directly?

tenderlove: Yeah it's supposed to give you access to the output socket IO.

miyagawa: Was that to implement something like WebSocket and HTTP/2.0

tenderlove: Yeah. There's a gem called faye, it's a websocket implementation for Rack, and it uses the hijack API and it just deals with the socket directly. So I wanna make it so that you can deal with the socket - well, excuse me, not deal with the socket directly but deal with something that looks like an IO. Right. Maybe under the hood it's not actually - you may not actually have direct access to the socket itself, but it will look like it from the API standpoint.

18:51

miyagawa: Yeah. One of the things I liked about Rack was - it's really simple. It's an env - you might think it looks like a global variable, and I think indeed it is, but it's an env hash, and all of the variables you need are already in there, and you can pull out all the information you need and then return an array with the status code, headers and body. I think that's a great, simple idea taken out of Python's WSGI I think. When I look at the_metal, just like you said, it's based on the object model - request object and response object. It's more like Node.JS - did you take an inspiration from there?

tenderlove: Oh absolutely. Yes, so basically, I mean - Um, don't tell anybody this, I used to be a J2EE developer (laughs) But basically, I thought it was weird going from J2EE to Rails and then you know, Rails is great, and then when we start using this env hash stuff in Rack, like "This is terrible. This is a step backwards from what I was doing in J2EE!" And then came along Node with ... I mean Node has a request and response object, not only do they have a request and response object but their API is dead simple. It's a very simple API. So the way I looked at theirs was basically, the servlet containers, but simple, like completely simplified. This is perfect. I don't want to know, we shouldn't have to know this giant API. Node has enough. It has everything we need, none of the other stuff, and I think we just steal from it.

miyagawa: Cool. Yeah when I first looked at Request/Response object, I immediately thought about Servlet APIs with hundreds of methods that adapters need to implement.

tenderlove: I don't want to do that (laughs)

miyagawa: (laughs)

tenderlove: I don't want to do that at all. Yeah the point is like, I love Node's API - it's terrible I'm a Rails developer and saying this. I love Node's API and I want to steal from it. I mean, it's important for us to be looking at other stuff, and stealing from them. So I want to steal from that, but I also want to make sure that it's going to work well with HTTP2. That's another focus of the project. It's like how we can make sure that - i want to make sure that what we'll release or what Rack 2.0 becomes will work very well with 1.1 servers, because that's what we have now. That's what we have to deal with today. Most of the servers are gonna be 1.1 for a long time. But I want to make sure that we can extend it to 2.0 servers easily, because 2.0 is the future.

21:44

miyagawa: What's the current target you want to work with in terms of the server implementations? I think, one of the most popular server implementations will be Unicorn, Puma and Thin. Are you targeting some of those, or all of those, as the experiment with the_metal goes?

tenderlove: Sure. I've already targeted Unicorn and Puma, and Webrick. So I have those three working right now. I haven't dealt with Thin yet, I'm not sure. For me, Thin will probably be a best effort, because I usually use Puma or Unicorn, and possibly Passenger. But I think the Phusion guys will probably - hopefully they'll take care of that for me(laughs).

miyagawa: When I looked at rack-hijack I think it was targeting Thin as well.

tenderlove: Was it?

miyagawa: Maybe. Thin actually has a secret key in the env, thin.async (async.callback) or something to implement websocket or something like that, even way before rack-hijack did.

tenderlove: Hmm, I didn't know that. See, this is what I'm saying about the env hash. There's secret key, like "secret keys." (laughs) Why do we have secret keys? Should just be methods, come on (laughs)

miyagawa: (laughs) Yeah.

23:06

miyagawa: When you implement asynchronous implementation for http2 or websocket, what's gonna be - I mean these servers have different event model - Puma will run on JVM with native threads and everything, and Thin has this EventMachine under the hood, and Unicorn will mostly (be) worker-based preforking servers. When you implement a real-time APIs on top of the Rack 2.0 or the_metal, what would your approach be?

tenderlove: So I think, for now, I mean this is getting into really a gray area, because I'm not totally sure. This is why it's experimental. But right now what I have in mind basically is, say depending on the HTTP 2.0 semantics, with the different ways your application can be served, we'll just have - you'll provide an object to the server, and that object has methods that the server might call, given the particular events for http/2.0. So probably, if you provide an object that only responds to "call" which is the current one, then we'll assume your app is an http/1.1 app. But if you respond to these other things then we'll upgrade you to 2. So that's like a very grey plan.

miyagawa: Interesting. If the listeners are interested in contributing to the_metal what's the great place to start?

tenderlove: Study HTTP 2.0. Figure out how we handle that well (laughs)

miyagawa: OK (laughs)

tenderlove: Because that's I need the most help. Yeah.

miyagawa: Yeah, cool.

25:05

miyagawa: Right - do you have any other things you want to talk about?

tenderlove: I don't know - Matz はWebサーバの話をしたい? (laughs)

miyagawa: Matz is here. (laughs)

tenderlove: You're not a web guy at all?

matz: No.

tenderlove: OK. I guess I'm a web guy. (laughs)

25:30

tenderlove: Ah, what should we talk about? We talked about ActiveRecord, we talked about themetal. Ah I will tell you why I called it themetal.

miyagawa: Yup.

tenderlove: OK.

miyagawa: I think the "Metal" is already used in the Rails terminology?

tenderlove: Yes, there's like ActionController::Metal. Yeah, so well this is a totally separate thing, so I called it "the" metal. That's the whole name for it. The reason I did that is so that, if you have fewer middleware, then your code is closer to the metal. (laughs)

miyagawa: (laughs)

tenderlove: So yes, it is a stupid joke (laughs)

miyagawa: It is a stupid joke, but used everywhere! Rails uses ActionController::Metal.

tenderlove: I know! (laughs) Why... Why do they say "closer to the metal" Are you closer to silicon? Doesn't make sense! There's like a very tiny amount of metal (laughs)

miyagawa: I know (laughs)

miyagawa: Yeah, Apple has released the iOS 8 API to the graphics library for the Apple's GPU and it's called the Metal as well.

tenderlove: The Metal? Oh really. See I would imagine if you're saying you're getting close to the metal, maybe the computer's case or something like that?

miyagawa: (laughs)

tenderlove: Why does that even mean CPU - it doesn't make sense (laughs) I don't know where this phrase comes from. (laughs)

miyagawa: So you dress like 80's heavy metal (band)

tenderlove: Yes exactly. It's really good because in English you can make so many ダジャレ (puns) with the metal. Really. Cause you know, Metal music, like Metal - just - getting close to the metal which is CPU, just so many different weird meanings. (laughs)

matz: Yeah, you can check out the BABYMETAL.

tenderlove: Ah yes! BABYMETAL. (laughs)

miyagawa: BABYMETAL.

tenderlove: Yes (laughs)

miyagawa: Close to the BABYMETAL (laughs) I think we've got a title for this episode.

tenderlove: "Close to the BABYMETAL" Perfect (laughs)

miyagawa: (laughs)

27:47

miyagawa: I think it's 30 minutes now, it's a great episode.

tenderlove: OK. So you're recording this? I thought it was live.

miyagawa: Yeah, I'm recording this live and I'll post this maybe later today.

tenderlove: So why is it called Rebuild.fm?

miyagawa: Hm, good question.

tenderlove: Are you rebuilding FM? (laughs)

miyagawa: (laughs) Rebuilding the radio industry.

tenderlove: I see, I see. But it's not FM right? It's podcast.

miyagawa: It's AM, more like.

tenderlove: Live.

miyagawa: It's live.

tenderlove: It's live, but on the internet, it's not like FM. it's not FM at all.

miyagawa: I don't know. (laughs) So some of the podcasters started taking these domains with .FM.

tenderlove: Ah. Is there .AM?

miyagawa: Yeah there is .AM as well.

tenderlove: You should get .AM. Rebuild.am too.

miyagawa: Yeah (laughs)

tenderlove: (laughs)

miyagawa: FM is really expensive. $75 or something.

tenderlove: Really.

miyagawa: Yeah, so I kind of hesitated to take the domain, but it's a really good domain. I like the name "Rebuild". Rebuild is like, "you have to rebuild the database, rebuild the code, entire project to rehash everything." It's a good name.

28:55

tenderlove: Did you know - so I tried to register a domain name with emoji in it lately. Did you try to do that? (laughs)

miyagawa: Yeah, Zak told me that he actually got an emoji domain with .hk or something. So yeah it's totally doable.

tenderlove: So I want to start - I've never written a programming language before, and I wanna write my own programming language..

matz: Yeah, do it.

tenderlove: Well I wanna call it poo-lang. Like the emoji poo-lang.

miyagawa: Pile of poo? (laughs)

tenderlove: Yes, exactly. So the thing is, everybody complains about programming languages, right. They'll be like "Oh Ruby is slow" and then you can say "Well have you looked at poo-lang? It's like the worst language!" (laughs)

miyagawa: (laughs)

tenderlove: So like, anytime anybody complains about some programming language, you can say "well just look at this one, this is the worst!" (laughs) So I wanna try implementing the worst programming language ever.

miyagawa: "My programming language is better than poo."

tenderlove: Yes! Everybody can say it.

matz: It's Shit! (laughs)

tenderlove: Yes! (laughs) So I want to try and get :poo:-lang.org but you can't register an emoji domain on org. I can't find a registrar that allows me to do it.

miyagawa: Maybe .io?

tenderlove: .io yes... (laughs)

miyagawa: That was a terrible joke.

tenderlove: (laughs) poo-lang.io. Yes. (laughs) Your code is terrible, what you wrote - the input is terrible, so is the output.

miyagawa: (laughs)

tenderlove: It works so well, cause you can say "well the GC is terrible, it's poo-lang, piece of poo." (laughs)

miyagawa: But you can flush it, flush the memory in one time.

tenderlove: Yeah. Just absolutely.

miyagawa: It's volatile...

tenderlove: OK. (laughs)

miyagawa: I mean, I should stop (making) terrible jokes.

tenderlove: Alright, that's it. Yes, so I'm announcing the development of poo-lang.

miyagawa: Is it on github?

tenderlove: Not yet. No. I've just been thinking about it.

miyagawa: Can you use emoji for a github project?

tenderlove: Yeah you can. I've done it. It shows up in the URL too, so you can put an emoji in URL (laughs) That works but you know what doesn't work is, you can't make a rubygem with emoji names. Because I tried to make - so all rubygems code works like you can package it. I made a gem called like I don't know hearts, a bunch of hearts, you can package the gem, but when you try to push it to rubygems.org, it rejects it.

miyagawa: Hmm.

tenderlove: So, the entire process up to pushing it works.

miyagawa: you just create a :poo:.rb file in the local filesystem and it works?

tenderlove: Yeah.

matz: It works.

tenderlove: Works fine. (laughs)

miyagawa: Nice. (laughs)

tenderlove: That's important! (laughs)

miyagawa: Great. Thanks for stopping by.

tenderlove: Yeah, thank you for having me.

miyagawa: Enjoy your talk and have a safe trip back to Seattle.

tenderlove: I will. Thank you.

miyagawa: Thank you very much.

-- Transcribed by miyagawa