> In interpreted languages I believe the thing that gets saved to mem is a pointer to the code itself. So no inherit difference here in terms of allocation. For closures reference counting is increased for the referenced variables but this is not a closure. ...
This has all been tread over before -- I won't rehash it since it seems like we're going in circles.
The hot path is worth optimizing first, in my mind -- giving up perf there for stats which will be called much less doesn't make sense to me.
To each their own!
> 1st. Sorting on the web API is definitively wrong. This is especially true in nodejs where the mantra is always non blocking code.
This assertion is so wide that it cannot be true. No matter what language you're in, you can absolutely do sorting in the API layer.
> 2nd. Pipeline should 100 percent be known. Atomic operations are 100 percent required across shared storage and it's expected to occur all the time. Database transactions are fundamental and not unexpected. Pipeline should be extremely common. This is not an obscure operation. I commented about it in my code in case the reviewer was someone like you who isn't familiar with how popular atomic database transactions are, but this operation is common it is not a obscure trick.
You don't need to pipeline if the operation is O(1) -- it's not a question of whether transactions are good/bad, I don't know why you're assuming people don't know about them.
It's the question of taking in unnecessary complexity here when there's an O(1) operation available. You're optimizing for an operation that is not on the hot path. The reason you have to do the atomic operation is because you're using an access pattern that requires it.
I know why and how you used an atomic transaction -- I disagree on whether it was necessary.
> That'd be stupid in my opinion. 302 redirect is something I expected. Maybe I should have stated that explicitly in the docs.
>
> What? In the specs all examples have trailing slashes. All of them.
This is correct, I read wrong and for that I apologize, they do have all trailing slashes. I was thinking of "/api/".
> Unlikely. Even as a lead it's better to cater to the majority opinion of the team. 99 percent of the time you never have full control.
>
> Developers and leads typically evolve to fit into the overall team culture while inserting a bit of their own opinions.
>
I referred to the context of the project -- you did* have full control.
If you were thinking of it in a team context, you should have asked more clarifying questions and found out more of what they wanted or would look for, similar to a real team context.
> No there isn't. You have to spin up redis. Flask documentation doesn't talk about that. Integration testing involves code that controls docker-compose. It's a bunch of hacky scripts with external process calls that are needed to get integration tests working.
>
> It's not rocket science but not within the scope of a take-home. Additionally hacking with docker compose is not sustainable or future proof, eventually you will hit infra that can't be replicated with docker.
>
> I will probably do it next time just to cater to people who share your viewpoints.
Again, this is a matter of what people think is hard and what isn't. Integration testing does not require that you spin up docker compose -- I literally linked to a libraries that do this in a reusable (importable) way w/ docker.
You said you don't do this kind of testing regularly, so it is understandable that it's not easy for you.
If docker was the problem, you can open subprocesses and assume that redis is installed in the test environment (i.e. your laptop, or CI environment), or take a binary to the redis binary path from ENV.
You can even remove the burden of running redis from the test suite and just make sure it's running manually or in test suite invocation/setup. There are many options, and it's just not that hard in 2023.
Don't know what (if anything) turned them off, so can't say this matters -- but it would matter to me, at the very least it would be a large plus for people who found the time to implement it.
> Lol. The prompt was write it in four hours ( no time) and they aren't paying me for it ( no money).
This is true, you did not have much time and they weren't paying you so I guess the situations are similar.
Seriously look at O(logN) and look at NlogN. Databases everywhere already take logN as acceptable. You are optimizing past that and introducing an NlogN sort step that will block all hot path calls because redis is single threaded. I don't think it's a to each their own thing here, i believe the user experience metrics will definitively play out towards my methodology but we can't know until we actually have metrics so we're at an impasse here.
>This assertion is so wide that it cannot be true. No matter what language you're in, you can absolutely do sorting in the API layer.
No this is absolutely true. Especially if you're a nodejs programmer. Nodejs is single threaded, sorting imposes NlogN penalty on a single thread.
Backend web development for the api layer is all about transmitting as much compute as possible to the database layer. You can get away with small sorts and things can still work but in general the practice of backend optimization is indeed to let the database handle as much as possible.
We can debate it but from my experience I know I'm pretty much categorically right on this. It's a fundamental principle. And it has special application to things that use Async IO like NodeJS. That's why there's huge emphasis in nodejs in writing non-blocking code. A sort is a potential block.
There are very few cases where you would handle compute on the api server rather then the database. The first instance is obvious, if N is guaranteed to be small which is not the case in the takehome. In other instances of sort with large N, the server ideally needs to launch that sort in a separate non-blocking thread so that the server can continue handling oncoming requests. In nodejs, to my knowledge since I last used it, you can't easily spawn a thread at will, you just have to pray you have available workers that aren't blocked by a sorting operation. In python the GIL makes this more complicated, but slightly better then nodejs.
For node you probably have 8 workers on 8 CPU cores. If you have 100 requests/s, all you need is 8 of them calling a sort on large N and all threads become blocked.
NlogN stands at the cusp here. It could work for small N, but for large N it will fail. Anything above NlogN is a huge risk.
>You don't need to pipeline if the operation is O(1) -- it's not a question of whether transactions are good/bad, I don't know why you're assuming people don't know about them.
This is categorically false. The pipeline operation as I said HAS NOTHING to do with O(1) or O(N).
This is essentially the operation:
set["key"] += 1
# another thread can update the value here leading to print displaying something unexpected.
print(set["key"])
So to solve the above comment you have to place both operations in a transaction. No two ways about it. The above operation is in spirit what I am doing. I update a value, then I read the value. Two operations and a possible race condition in between.
>Again, this is a matter of what people think is hard and what isn't. Integration testing does not require that you spin up docker compose -- I literally linked to a libraries that do this in a reusable (importable) way w/ docker.
You linked? Where? I'd like to know about any library that will do this. Tell me of any library that does integration tests that spins up infrastructure for you. The only one closest I can think of that you can run locally is anything that would use docker-compose or some other IAC language that controls containers. I honestly don't think any popular ones exist.
>If docker was the problem, you can open subprocesses and assume that redis is installed in the test environment (i.e. your laptop, or CI environment), or take a binary to the redis binary path from ENV.
No way I'm going to assume the user has redis installed on his local machine. Many devs don't. It's all remote development for them or everything lives in containers.
Docker is not the problem. Docker or virtual machines makes this problem more amenable to a solution, but even using docker here with testing is overkill and hacky. A take home should not expect a user to build excessive infrastructure locally just to run tests.
>You said you don't do this kind of testing regularly, so it is understandable that it's not easy for you.
I'd like to know your perspective. How is spinning up infrastructure for integration tests easy? You have to write code that launches databases, launches servers and views the entire infra externally. Do-able? Yes? Easy? depends on what you call easy. As easy as unit tests? Definitely no. Easy enough for a take home? In my opinion, no.
Maybe for this take home project you could be right. I could do some integration tests by spinning up docker-compose from within python. Hacky but doable. But in general this solution is not production scalable as production involves more things then what can be placed inside a docker-compose.
>You can even remove the burden of running redis from the test suite and just make sure it's running manually or in test suite invocation/setup. There are many options, and it's just not that hard in 2023.
So I should expect the reviewer to have redis running on his local machine? I would expect in 2023, I should have the entire project be segregated from the context it's running in. That's the cutting edge I believe.
Yeah it's 2023, you tell me how integration testing should be done as easily as unit testing on a takehome. I had one takehome project involving S3 and aws lambdas. They expected me to get an AWS account and literally set up infrastructure because there's no way around even testing what I wrote without actual infrastructure. That entire project was just integration test nightmare. Much rather run asserts locally.
>I referred to the context of the project -- you did* have full control.
Well I mean 99% of hires get into a context where they aren't in full control. So why is it logical to test what a developer will do if he/she had full control? Isn't it better to test the developers ability to code and to adapt to different contexts? Your methodology sort of just tests the programmers personal philosophy and whether it matches your own. That was my point.
> You aren't getting what I'm saying. logN is already a huge optimization for a hot path. Look at that pic: https://dev-to-uploads.s3.amazonaws.com/i/ya7xk4n9ch50xgwibt...
>
> Seriously look at O(logN) and look at NlogN. Databases everywhere already take logN as acceptable. You are optimizing past that and introducing an NlogN sort step that will block all hot path calls because redis is single threaded. I don't think it's a to each their own thing here, i believe the user experience metrics will definitively play out towards my methodology but we can't know until we actually have metrics so we're at an impasse here.
Just to keep the comparison in line -- we're comparing O(log(N)) @ O(1) in redis-land.
On the stats side we're comparing O(N) on the redis side PLUS either O(Nlog(N)) or O(N) on the python side.
> No this is absolutely true. Especially if you're a nodejs programmer. Nodejs is single threaded, sorting imposes NlogN penalty on a single thread.
NodeJS has both child processes and workers for heavy computation:
((this is also part of the reason I think NodeJS is strictly superior to other "scripting" languages as mentioned briefly in the other comment))
Again, here you seem to be arguing against a strawman that doesn't know that blocking the IO loop is bad. Try arguing against one that knows ways to work around that. This is why I'm saying this rule isn't true. Extensive computation on single-threaded "scripting" languages is possible (and even if it wasn't, punt it off to a remote pool of workers, which could also be NodeJS!).
All of this is a premature optimization, but my point here is that the stats can be optimized later. You could literally compute it periodically or continuously and only return the latest cached version w/ a timestamp. This makes the stats endpoint O(1).
> This is categorically false. The pipeline operation as I said HAS NOTHING to do with O(1) or O(N).
> ...
I think this is where we're talking past each other, so let me explain more of how I see the problem -- the solution I have in mind is serializing the URL and using ONE call to INCR (https://redis.io/commands/incr/) on the ingest side.
There is a lot you can do with the data storage pattern to make other operations more efficient, but on the stats side, the most basic way you can do it is to scan
I will concede that given that we know the data should fit in memory (otherwise you just crash) your approach gives you O(N) retrieval time and it's definitely superior to not have to do that on the python side (and python just streaming the response through). I am comfortable optimizing in-API computation, so I don't think it's a problem.
Here's what I mean -- you can actually solve the ordering problem in O(N) + O(M) time by keeping track of the max you've seen and building a sparse array and running through every single index from max to zero. It's overkill, but it's generally referred to as a counting sort:
This is overkill, clearly, and I will concede that ZSET is lighter and easier to get right than this.
> You linked? Where? I'd like to know about any library that will do this. Tell me of any library that does integration tests that spins up infrastructure for you. The only one closest I can think of that you can run locally is anything that would use docker-compose or some other IAC language that controls containers. I honestly don't think any popular ones exist.
I am very sure that I linked that, but in the case I didn't, here it is again -- hope you find it useful.
> No way I'm going to assume the user has redis installed on his local machine. Many devs don't. It's all remote development for them or everything lives in containers.
>
> Docker is not the problem. Docker or virtual machines makes this problem more amenable to a solution, but even using docker here with testing is overkill and hacky. A take home should not expect a user to build excessive infrastructure locally just to run tests.
I don't think these statements make sense -- having docker installed and having redis installed are basically equivalent work. At the end of the day, the outcome is the same -- the developer is capable of running redis locally. Having redis installed on your local machine is absolutely within range for a backend developer.
Also, remote development is not practiced by many companies -- the only companies I've seen doing thin-clients that are large.
> Maybe for this take home project you could be right. I could do some integration tests by spinning up docker-compose from within python. Hacky but doable. But in general this solution is not production scalable as production involves more things then what can be placed inside a docker-compose.
I see it as just spinning up docker, not compose -- you already have access to the app (ex. if it was buildable via a function) so you could spawn redis in a subprocess (or container) on a random port, and then spawn the app.
I agree that it is not trivial, but the value is high (in mymind.
> Yeah it's 2023, you tell me how integration testing should be done as easily as unit testing on a takehome. I had one takehome project involving S3 and aws lambdas. They expected me to get an AWS account and literally set up infrastructure because there's no way around even testing what I wrote without actual infrastructure. That entire project was just integration test nightmare. Much rather run asserts locally.
I agree that integration testing is harder -- I think there's more value there.
That said I do think that's a weakness of the platform compute stuff -- it is inconvenient to test lambda outside of lambda.
> Well I mean 99% of hires get into a context where they aren't in full control. So why is it logical to test what a developer will do if he/she had full control? Isn't it better to test the developers ability to code and to adapt to different contexts? Your methodology sort of just tests the programmers personal philosophy and whether it matches your own. That was my point.
Ah, this is true -- but I think this is what people are testing in interviews. There is a predominant culture/shared values, and the test is literally whether someone can fit into those values.
Whether they should or should not be, that's at least partially what interviews are -- does the new team member feel the same way about technical culture currently shared by the team.
Now in the case of this interview your solution was just fine, even excellent (because you went out of your way to do async io, use newer/easier packaging methodologies, etc), but it's clearly not just that.
>Again, here you seem to be arguing against a strawman that doesn't know that blocking the IO loop is bad. Try arguing against one that knows ways to work around that. This is why I'm saying this rule isn't true. Extensive computation on single-threaded "scripting" languages is possible (and even if it wasn't, punt it off to a remote pool of workers, which could also be NodeJS!).
Very rare to find a rule that's absolutely true.. I clearly stated exceptions to the rule (which you repeated) but the generality is still true.
Threading in nodejs is new and didn't exist since the last time I touched it. It looks like it's not the standard use case as google searches still have websites with titles saying node is single threaded everywhere. The only way I can see this being done is multiple Processes (meaning each with a copy of v8) using OS shared memory as IPC and they're just calling it threads. It will take a shit load of work to make v8 actually multi-threaded.
Processes are expensive so you can't really follow this model per request. And we stopped following threading per request over a decade ago.
Again these are exceptions to the rule, from what I'm reading Nodejs is normally still single threaded with a fixed number of worker processes that are called "threads". Under this my general rule is still generally true: backend engineering does no typically involve writing non blocking code and offloading compute to other sources. Again, there are exceptions but as I stated before these exceptions are rare.
>Here's what I mean -- you can actually solve the ordering problem in O(N) + O(M) time by keeping track of the max you've seen and building a sparse array and running through every single index from max to zero. It's overkill, but it's generally referred to as a counting sort:
Oh come on. We both know these sorts won't work. These large numbers will throw off memory. Imagine 3 routes. One route gets 352 hits, another route gets 400 hits, and another route gets 600,000 hits. What's Big Oh for memory and sort?
It's O(600,000) for both memory and runtime. N=3 and it doesn't even matter here. Yeah these types of sorts are almost never used for this reason, they only work for things with smaller ranges. It's also especially not useful for this project. Like this project was designed so "counting sort" fails big time.
Also we don't need to talk about the O(N) read and write. That's a given it's always there.
>I don't think these statements make sense -- having docker installed and having redis installed are basically equivalent work. At the end of the day, the outcome is the same -- the developer is capable of running redis locally. Having redis installed on your local machine is absolutely within range for a backend developer.
Unfortunately these statements do make sense and your characterization seems completely dishonest to me. People like to keep their local environments pure and segregated away from daemons that run in a web server. I'm sure in your universe you are claiming web developers install redis, postgresql and kafka all locally but that just sounds absurd to me. We can agree to disagree but from my perspective I don't think you're being realistic here.
>Also, remote development is not practiced by many companies -- the only companies I've seen doing thin-clients that are large.
It's practiced by a large amount and basically every company I've worked at for the past 5 years. Every company has to at least partially do remote dev in order to fully test E2E stuff or integrations.
>I see it as just spinning up docker, not compose -- you already have access to the app (ex. if it was buildable via a function) so you could spawn redis in a subprocess (or container) on a random port, and then spawn the app.
Sure. The point is it's hacky to do this without an existing framework. I'll check out that library you linked.
>I agree that integration testing is harder -- I think there's more value there.
Of course there's more value. You get more value at higher cost. That's been my entire point.
Good finds. But what about SNS, IOT, Big Query and Redshift? Again my problem isn't about specific services, it's about infra in general.
>Ah, this is true -- but I think this is what people are testing in interviews. There is a predominant culture/shared values, and the test is literally whether someone can fit into those values.
No. I think what's going on is people aren't putting much thought into what they're actually interviewing for. They just have some made up bar in their mind whether it's a leetcode algorithm or whether the guy wrote a unit test for the one available pure function for testing.
>Whether they should or should not be, that's at least partially what interviews are -- does the new team member feel the same way about technical culture currently shared by the team.
The answer is no. There's always developers who disagree with things and just don't reveal it. Think about the places you worked at. Were you in total agreement? I doubt it. A huge amount of devs are opinionated and think company policies or practices are BS. People adapt.
>Now in the case of this interview your solution was just fine, even excellent (because you went out of your way to do async io, use newer/easier packaging methodologies, etc), but it's clearly not just that.
The testing is just a game. I can play the game and suddenly I pass all the interviews. I think this is the flaw with your methodology as I just need to write tests to get in. Google for example in spirit attempted another method which involves testing IQ via algorithms. It's a much higher bar
The problem with google is that their methodology can also be gamed but it's much harder to game it and often the bar is too high for the actual job the engineer is expected to do.
I think both methodologies are flawed, but hiring via ignoring raw ability and picking people based off of weirdly specific cultural preferences is the worse of the two hiring methodologies.
Put it this way. If a company has a strong testing culture, then engineers who don't typically test things will adapt. It's not hard to do, and testing isn't so annoying that they won't do it.
> Very rare to find a rule that's absolutely true.. I clearly stated exceptions to the rule (which you repeated) but the generality is still true.
>
> Threading in nodejs is new and didn't exist since the last time I touched it. It looks like it's not the standard use case as google searches still have websites with titles saying node is single threaded everywhere. The only way I can see this being done is multiple Processes (meaning each with a copy of v8) using OS shared memory as IPC and they're just calling it threads. It will take a shit load of work to make v8 actually multi-threaded.
>
> Processes are expensive so you can't really follow this model per request. And we stopped following threading per request over a decade ago.
>
> Again these are exceptions to the rule, from what I'm reading Nodejs is normally still single threaded with a fixed number of worker processes that are called "threads". Under this my general rule is still generally true: backend engineering does no typically involve writing non blocking code and offloading compute to other sources. Again, there are exceptions but as I stated before these exceptions are rare.
You seem to always fight the least knowledgeable strawmen. Process pooling and task distribution exist. Just because you didn't know about worker threads in NodeJS has nothing to do with me. Whether it's common or not has nothing to do with the validity of the solution.
Yet another reason why NodeJS is superior as a platform to CPython.
Anyway, I will take that you have relaxed your rule -- it needed to be relaxed. IIRC you said it was "definitively true" or something to that effect.
> Oh come on. We both know these sorts won't work. These large numbers will throw off memory. Imagine 3 routes. One route gets 352 hits, another route gets 400 hits, and another route gets 600,000 hits. What's Big Oh for memory and sort?
The array is sparse, and yes, this is exactly what I expect to happen. I wrote the scenario thinking of that.
> It's O(600,000) for both memory and runtime. N=3 and it doesn't even matter here. Yeah these types of sorts are almost never used for this reason, they only work for things with smaller ranges. It's also especially not useful for this project. Like this project was designed so "counting sort" fails big time.
>
> Also we don't need to talk about the O(N) read and write. That's a given it's always there.
Modern processors churn through numbers pretty quick, 600,000 operations is not a big deal -- still O(N)!
It's better to be clear than unclear -- it was a cause for confusion, evidently because you were comparing O(Log(N)) and O(NLog(N)). The comparison for ingest O(Log(N)) and O(1).
This is the last I'm going to say on this.
> Unfortunately these statements do make sense and your characterization seems completely dishonest to me. People like to keep their local environments pure and segregated away from daemons that run in a web server. I'm sure in your universe you are claiming web developers install redis, postgresql and kafka all locally but that just sounds absurd to me. We can agree to disagree but from my perspective I don't think you're being realistic here.
In the world before docker and docker compose, this is what people did when they had to run software to test locally. In the current world, people still do this, but it's made easier by docker and docker-compose.
That's the last I'm going to say on that.
> Sure. The point is it's hacky to do this without an existing framework. I'll check out that library you linked.
God forbid we write useful code without an existing framework.
> Of course there's more value. You get more value at higher cost. That's been my entire point.
Except "cost" here is relative -- not everyone considers that cost to be high. Relative to unit tests it is higher, but it is still trivial with knowledge.
It's simple, you just didn't have the knowledge so you thought it was prohibitively hard. That's fine -- that's just how technology goes.
> Good finds. But what about SNS, IOT, Big Query and Redshift? Again my problem isn't about specific services, it's about infra in general.
The goalposts seem to be moving, so time for me to get off the field. You noted those other tech specifically, so I answered.
Read up on ways you can replicate these environments locally, or don't. If you have some you can't replicate, don't use them, or find other ways to test them that aren't completely local (i.e. running in a test account).
That's the last I'm saying on that.
> No. I think what's going on is people aren't putting much thought into what they're actually interviewing for. They just have some made up bar in their mind whether it's a leetcode algorithm or whether the guy wrote a unit test for the one available pure function for testing.
Alright well I hope you're able to find a way through the minefield of reality, either way that's the last I'm saying on it.
> The testing is just a game. I can play the game and suddenly I pass all the interviews. I think this is the flaw with your methodology as I just need to write tests to get in. Google for example in spirit attempted another method which involves testing IQ via algorithms. It's a much higher bar
>
> The problem with google is that their methodology can also be gamed but it's much harder to game it and often the bar is too high for the actual job the engineer is expected to do.
>
> I think both methodologies are flawed, but hiring via ignoring raw ability and picking people based off of weirdly specific cultural preferences is the worse of the two hiring methodologies.
>
> Put it this way. If a company has a strong testing culture, then engineers who don't typically test things will adapt. It's not hard to do, and testing isn't so annoying that they won't do it.
Well, don't expect to win the game if you don't play.
Hope this was illustrative of at least how others think -- this is the last I'm going to post, thanks for the discussion!
This has all been tread over before -- I won't rehash it since it seems like we're going in circles.
The hot path is worth optimizing first, in my mind -- giving up perf there for stats which will be called much less doesn't make sense to me.
To each their own!
> 1st. Sorting on the web API is definitively wrong. This is especially true in nodejs where the mantra is always non blocking code.
This assertion is so wide that it cannot be true. No matter what language you're in, you can absolutely do sorting in the API layer.
> 2nd. Pipeline should 100 percent be known. Atomic operations are 100 percent required across shared storage and it's expected to occur all the time. Database transactions are fundamental and not unexpected. Pipeline should be extremely common. This is not an obscure operation. I commented about it in my code in case the reviewer was someone like you who isn't familiar with how popular atomic database transactions are, but this operation is common it is not a obscure trick.
You don't need to pipeline if the operation is O(1) -- it's not a question of whether transactions are good/bad, I don't know why you're assuming people don't know about them.
It's the question of taking in unnecessary complexity here when there's an O(1) operation available. You're optimizing for an operation that is not on the hot path. The reason you have to do the atomic operation is because you're using an access pattern that requires it.
I know why and how you used an atomic transaction -- I disagree on whether it was necessary.
> That'd be stupid in my opinion. 302 redirect is something I expected. Maybe I should have stated that explicitly in the docs. > > What? In the specs all examples have trailing slashes. All of them.
This is correct, I read wrong and for that I apologize, they do have all trailing slashes. I was thinking of "/api/".
> Unlikely. Even as a lead it's better to cater to the majority opinion of the team. 99 percent of the time you never have full control. > > Developers and leads typically evolve to fit into the overall team culture while inserting a bit of their own opinions. >
I referred to the context of the project -- you did* have full control.
If you were thinking of it in a team context, you should have asked more clarifying questions and found out more of what they wanted or would look for, similar to a real team context.
> No there isn't. You have to spin up redis. Flask documentation doesn't talk about that. Integration testing involves code that controls docker-compose. It's a bunch of hacky scripts with external process calls that are needed to get integration tests working. > > It's not rocket science but not within the scope of a take-home. Additionally hacking with docker compose is not sustainable or future proof, eventually you will hit infra that can't be replicated with docker. > > I will probably do it next time just to cater to people who share your viewpoints.
Again, this is a matter of what people think is hard and what isn't. Integration testing does not require that you spin up docker compose -- I literally linked to a libraries that do this in a reusable (importable) way w/ docker.
You said you don't do this kind of testing regularly, so it is understandable that it's not easy for you.
If docker was the problem, you can open subprocesses and assume that redis is installed in the test environment (i.e. your laptop, or CI environment), or take a binary to the redis binary path from ENV.
You can even remove the burden of running redis from the test suite and just make sure it's running manually or in test suite invocation/setup. There are many options, and it's just not that hard in 2023.
Don't know what (if anything) turned them off, so can't say this matters -- but it would matter to me, at the very least it would be a large plus for people who found the time to implement it.
> Lol. The prompt was write it in four hours ( no time) and they aren't paying me for it ( no money).
This is true, you did not have much time and they weren't paying you so I guess the situations are similar.