Skip to the posts
Buffalo's neighborhood journals since 2003 🍀 Patrick Spicy Devil Logan🌭 Sahlen🥨 Weber Spank🥟 Pierogi🐸 Scaj🍺 Labatt Utica Pink Cherio❄️ Snow Murder

tinypliny’s journal

Computing frustrations

·

I just need a computing platform to perform many operations (~2000) on a small dataset in a reasonable amount of time without dragging my system to a standstill every time. Is that too much to ask? Apparently, it is. The options I have are dismal.

- Get a computing account at UB's center for computational research. This is easier said than done. There is so much bureaucratic red-tape and craziness involved in just getting a puny account that it makes sense at this point to accept that I will never be able to use their resources.

- Buy a laptop that will run linux and then R on top of it efficiently utilizing all the processing cores and the memory. This is a difficult proposition because R doesn't utilize all cores natively. It's a pain to figure out how to set up linux to utilize all cores with R and make the processing efficient - with my minimal level of expertise. So I am stuck with a spiked up CPU load and system freezes every time I run an analyses. Should I be spending close to $1.5k for such misery. In the end, maybe I will have to.

- Get a mac. NO. Their hardware is appalling. They get away with such mediocrity because of their sheep fanboy base that is blind to the fact that their hard-drives and memory fail with the frequency of a rainstorm in an Amazon forest. I like well-functioning software but if I have to be queuing up at some service center (however fanciful its name) for repairs or replacements every few months, I don't think I will ever wander close to such a product.

I wish things were not this complicated.
Share 286 words · edited Feb 9, 2011

💬 4 comments

  1. paul ·

    Is it possible that its your prog? 2000 calculations is not that crazy. How big is the dataset? Perhaps you can find ways to make it more efficient.
  2. carolinian ·

    StackOverflow is a great place to get answers to these kinds of things.
    :::link:::

    While the easy solution is to throw more hardware at it, if that's not available, you could spend some neurons and learn C and write the program in C, and you could probably get it done all on one core with your old hardware and save some $$$.

  3. tinypliny ·

    Hmm... I checked my code again. I don't know if I have enough knowledge of S+ to tweak my code so much. @carolinian - I found some useful answered questions at StackOverflow. Thanks! There is a package that makes R use multiple cores under ubuntu. I am going to take the fedora out and put ubuntu into my main laptop and give the data-crunching another whirl (its nearly 400 X 15 data points - not really a huge dataset).

    Learning how to write code in C will not really solve my problem. I fear I will be stuck in non-progress mode for ever. R is based on S+ and many PhDs (in statistics/biostatistics/systems engineering) across the world write packages that carry out algorithms and models for R. If I go the C route, not only will that involve learning how to code all those algorithms but also code the options for the algorithms and all the underlying basic statistical procedures that underlie each statistical model. In a sense, it will be like learning to write an equivalent of the R framework, its packages, its command-options AND its code in C or C++ - that is something so daunting, I don't even want to go near that solution! :) I am sure if you made a statistical package repository based on C, they will name it after you! :)
  4. tinypliny ·

    Aarrgh, I wish computing power were not so expensive...

💬 Write a comment