A Word to the Wise

Had a real laugher/shocker at the same time. I develop so much App code where the data processing is a flow and not a stop:persist:process:continue action that sometimes you get burned by that. Everything was working fine as "hey download all that shop's orders, parse them, deal with the contents, and move on". Everything as working fine on MY itty, bitty, little dev shop.

One of my clients decided to take my dare and hook-up. Nothing good came of that. I checked my App logs and found lots of nice horror. R14 out of memory, bad buffers, failed floobles... ya baby... A quick code inspection showed me I was downloading 50,000 orders to make somewhere between 50,000 and 150,000+ new DB rows. Thing is I stupidly wrote my loop getting those 50,000 API orders as one that stored them in ALL their entirety as objects in memory. Nice!! Except my little cloud process has only 500 MB allocated to it. 50,000 full orders will almost certainly squeeze/bust that limit quicker than you can shake a stick. All I needed was the 50,000 ID's and an array of 50,000 integers can easily fit in memory like that A-OK.

So the takeaway is to be careful when inviting your clients to try out something where the size of their shop's business can crush your App to a pulp in no time.