I made the mistake of thinking that client.run() would run a function on each Dask worker regardless if the worker hadn't started yet. In the case where new workers come online that function won't run. Instead you'll need to take advantage of the register_worker_callbacks() function that will register a function to run on all of the workers at startup. It looks like this:
cluster = LocalCluster()
client = Client(cluster)
client.register_worker_callbacks(setup=your_function_name)
I found this by looking through the tests for this function's pull request. It's otherwise undocumented.
A blog about scientific computing with Python and Matlab. See the work of an engineer and data scientist in practice.
Friday, March 22, 2019
Friday, March 15, 2019
Salary Progression (in Tech)
This was an insightful article on one person's salary progression in technology. Averages out to be something like $11,000 per year in increase over the 15 year career so far. This matches pretty well with other folks I've talked to. An accompanying Hackernews post has more datapoints. Stock options / RSUs tend to skew things and lend a high degree of variability to compensation. At higher levels it isn't uncommon for a large portion of your total compensation to be in RSUs.
Of course, where you live causes a wild variation in the value of your income. Additionally, if you're working for dope cash but miserable it's hard to see that as a net positive.
Of course, where you live causes a wild variation in the value of your income. Additionally, if you're working for dope cash but miserable it's hard to see that as a net positive.
Thursday, March 14, 2019
New Job For 2019
I'm now working as a Senior Machine Learning Engineer for Grubhub on the order volume forecasting team. We own predictive time series models that produce forecasts the business uses to schedule drivers.
The job search took about 3 months and involved at least 5 out-right rejections, a lot of non-answers, and 2 offers.
The job search took about 3 months and involved at least 5 out-right rejections, a lot of non-answers, and 2 offers.
Subscribe to:
Posts (Atom)