Welcome to the Treehouse Community

Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.

Looking to learn something new?

Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.

Start your free trial

Ruby

Rob Munoz
PLUS
Rob Munoz
Courses Plus Student 586 Points
Posted by Rob Munoz
Rob Munoz
Courses Plus Student 586 Points

Large .txt Processing With Ruby

I have a large .txt file that needs parsing in Ruby (or any language for that matter). How do people get over the string size limitations or handle large objects in Ruby.

Best Regards!

  • Rob M.
Brandon Keene
Brandon Keene
7,217 Points
Brandon Keene
Brandon Keene
7,217 Points

Rob Munoz

I took a look at that article too, but I walked away from it feeling like "OK, so there are performance gains if you keep your string below 23 characters, but often that may be an unattainable goal." And since he doesn't really offer up much in the way of alternatives, I chalked it up to a sort of "here's a fun, unavoidable fact about Ruby performance" article. Unless your file is absolutely massive, nothing I read suggested to me that Ruby would be truly incapable of handling it.

Happy to help, and best of luck parsing that file!

1 Answer

Brandon Keene
Brandon Keene
7,217 Points
Brandon Keene
Brandon Keene
7,217 Points

I wasn't sure whether or not Ruby had char limits on strings, so you got me doing a bit of googling. Now my understanding is nebulously "Yes, sort of, but they need to be huge."

I found this article that seems like it might be able to address your question: http://stackoverflow.com/questions/4846394/how-to-efficiently-parse-large-text-files-in-ruby

Rob Munoz
Rob Munoz
Courses Plus Student 586 Points
Rob Munoz
Rob Munoz
Courses Plus Student 586 Points

Thanks Brandon! I was under that same impression originally, but after my google attempt I came away somewhat discouraged by this:

http://patshaughnessy.net/2012/1/4/never-create-ruby-strings-longer-than-23-characters

I guess if you're talking about large string processing, "large" is really relative to the application. In Pat's post he mentions (I may be taking this out of context), for strings > 23 char in length, 1 million strings are created at 500 ms vs 250. This seems like a big deal but it might not be in my current project.

I posted in his blog asking about an alternative to storing data more efficiently but could be a while/never before I hear back.

Thanks so much for the input.