I disagree.

* I routinely find people using the wrong data structure, when there exists a better one, with better O() time/space.

* I find people tend to not understand BTrees, particularly when there are two attributes being indexed. Given an index on (a, b), I find it common misconception that the BTree can efficiently answer `$a_min < a < $a_max AND $b_min < b < $b_max`. (I.e., people do not understand that the tree cannot make use of the second < condition, and must scan potentially many more rows than they intend.)

* Graph theory. git uses it. Any sort of dependency tree uses it.

That said, I acknowledge that software engineering does require a lot of non-theoretical knowledge, which is why I ask both types of questions in an interview.

That's where I disagree. For 90% of the code it wouldn't ever matter if you're using a list with O(n) or an array with O(1). And in the 10% it can usually be remedied afterwards. Usually the reviewer will point it out and it's a 10min fix.

On the other hand I've seen so much hand-optimized code that used the correct data structures to solve the wrong problem.

Yes, that is kind of unfair - you want people who grasp the problem and find a good solution. But I prefer the people who solve the problem at hand in a wrong way instead of solving the wrong problem in the best imaginable way.

> Usually the reviewer will point it out and it's a 10min fix.

And in order for the reviewer to recognize the situation at all, and then point it out, they would need knowledge of the underlying data structures, their performance, and as you mention, pragmatism to know when to use it.

(I'm presuming that everyone on your team shares the responsibility of code review, and you don't funnel it through a few people with knowledge of how things work. (I believe that robs those that don't of the opportunity to learn through review.))

You're absolutely right. But there's a difference between "spotting an error" and "being able to construct $datastructure_I_use_once_a_year" in an interview

Yeah, but data structures and binary trees are about as much as you really need to get. There's a big difference between selecting the right data structure and re-implementing dijkstra's work on a whiteboard.

Counterpoint - I routinely see people ignore the actual real life cost of pointer chasing because O(n) (you don't need a linked list if you only have 10 small elements).

I am assuming you are talking about database indexes. The documentation for the database that I have most used actually specified what the various supported types were good for. It made choosing the correct type of index much easier. The documentation may be sufficient for determining which index types will perform well for the expected queries, even if knowing detailed information about data structures is nice to know. Having documentation which describes the data structures well enough is really nice though, and should be encouraged a lot.

Does your job actually require knowledge of BTrees?

Yes. They're the typical data structure backing most relation database indexes. Knowing when they will perform well (and more often, when they won't) follows directly from knowing their structure. (The example of trying to do a range search on two dimensions in the post above is an example that doesn't perform as well as — again, I find — people naively expect it to.)

Have I ever implemented one? Not yet. Do I ask for a BTree implementation or exact, low-level understanding on an interview? No.

Yes! Our database blew up in our face because we misused and had wrong performance assumptions some indexes. Colleague had knowledge of btrees (the underlying data structure of database, iirc) and database internals. Optimised the whole thing. Queries are now 500x faster

Computer engineer != programmer.

1. Computer engineer is someone who knows computer science. Acquires knowledge that survives when tools die. Hired by Google and other firms.

2. Programmer. Blue collar worker writing bean counting programs. Mainly writes customer software for order. Gets shit done. Becomes difficult to employ when he turns 40 if the shit he does is outdated.

Google and the majority of silicon valley grossly abuse the term software engineer (which is what you describe). There are very few software engineers at google, mostly just developers, many of whom are actually PhDs, which is also not an engineer.

Engineering is about process and responsibility. A professional engineer is often a protected term, and comes with a certain legal authority to approve designs as sound and correct. The purpose of this is to give people a reasonably reliable way to ensure that the people they go to for engineering designs meet a minimum level of professional knowledge. AFAIK in some cases the engineer can be held liable if their design fails causing death.

Can you imagine if developers had to take on some liability by legally and officially "approving" that a software design meets some criteria of being "free from error"? It's almost crazy to consider it, though you can imagine that such a thing would actually be pretty important for say, medical equipment software, or aircraft control software. Unfortunately this is pretty far from the reality of software engineering.

Unlike other forms of engineering, there is no such thing as a "professional software engineer", though you can get a degree in the _topic_ of software engineering.

TLDR there are no "software engineers" in the same sense that you have electrical or mechanical engineers. The protection on the term engineer in software is commonly contested, though perhaps less so in the US then elsewhere.

I agree with you that "Software Engineer" is often used pretentiously. Supposedly a lot of North American companies advertise "Software Engineers" because the job category "engineer" is mentioned in the NAFTA treaty (i.e. it's easier to hire people that way).

Funny, in Canada they appear to do the exact opposite because engineering jobs are regulated. So instead of "software engineers" it's more common to find "software developers".

Most electrical and mechanical engineers aren't Professional Engineers either. A Professional Engineer's sign off is only needed on safety critical projects, so most don't bother with it because they don't work on safety critical projects.

A "computer engineer" is someone who designs microprocessors and physical computational hardware - not software. (And for the laity, it is not someone who fixes your broken desktop PC either).

Even if you meant SE instead of CE, I feel your distinction is arbitrary and almost classist - while it's true that the top notch folks hired by Google and Microsoft could write "bean counting programs" its fallacious to assume the contrarywise - major accounting and business software firms are just as picky when it comes to hiring - similarly I know plenty of small startups writing exotic software that are able to ship without needing to hire everyone from MIT an Stanford - I also know plenty of very intelligent and capable minds going to waste at companies like Google, MSFT and Facebook working on projects they dislike or for low-impact internal systems - while their contemporaries who went to a coding boot camp got picked up by Snapitterbook and become hot stuff despite never having read the Mythical Man-Month.

Computer Engineer != computer scientist.

1. Computer engineer works at and around the boundary between electrical engineering and computer science. Hardware, firmware, drivers, and other low-level systems that interact closely with the physical world. Hired by Intel and other firms.

2. Computer scientist knows the study of computation. Computer science tends to abstract the hardware away and consider ideal systems.

There is of course some overlap. Computer engineering is all about the overlap between CS and EE, so some CS people will work in some of the same areas as CEs. Likewise with EEs. And the terms are fuzzy, and may have different definitions to different people, but the distinction I made seems to be present in most college degree programs I've seen.

Assuming you meant "Software Engineer" not "Computer Engineer".

>> Acquires knowledge that survives when tools die

I would actually agree with this, the kicker is though this "acquired knowledge" is not CS knowledge - it's understanding of how to get shit done in the "real world" and is not something you can easily acquire by reading a CS book.

> So unless you spend all day writing compilers from scratch or calculating Pascal's Triangle, please stop with the ridiculous CS questions in interviews.

> Software Engineering is more of a trade, and requires vocational knowledge and experience. A mountain of theory may not always be required to Get Shit Done.

First, interview questions are engineering problems that either applicable to real world systems, or were inspired from them.

Second, interview questions are used to weed away candidates. At certain point, there will be a few candidates that can answer the questions and articulate sound engineering process. And that's one got hired.

"Implement a recursive depth first traversal of a tree" isn't an engineering problem. It's a CS textbook problem. A fairly basic one, surely, but about as useful to a software engineering project in most cases as the ability to smelt and construct a screw is to an automobile engineer. And the software interviews GP complained about these days don't even start with something that trivial. Usually it's more absurd, and involves what are essentially little math tricks.

> but about as useful to a software engineering project in most cases as the ability to smelt and construct a screw is to an automobile engineer

I strongly disagree. Having a deeper understanding of how the data structures you use day-to-day (whether or not you implemented them or not) is vital to you picking the correct structure for any particular task. Not knowing the basics of how to implement a binary tree, a singly or doubly linked list is a massive red flag in terms of the competence of a candidate in my humble opinion.

I think absolutely there are classes of data structures that are obscure or leftfield that shouldn't be asked of a candidate, but there's a hell of a lot that a professional should know, regardless of whether they're using a library that does it for them.

> Not knowing the basics of how to implement a binary tree, a singly or doubly linked list is a massive red flag in terms of the competence of a candidate in my humble opinion.

I agree.

On the flip side-- upon successfully implementing a linked list, a clever candidate might let it slip that he/she designed the interface to be performant with a particular use case in mind, and that they weren't sure if the interview had the same expectation for that use case. If the interviewer consequently signaled agreement with the premise that linked lists have performance benefits in certain cases, it's a good indicator that the people in charge probably aren't thinking critically about the software they are producing.

Is having a deeper than big-O understanding of data structure truely necessary?

A mechanical engineer will have an understanding the characteristics of different screws (tensile strength, corrosion resistance, material compatibly etc) enabling them to pick the right one for the task at hand. Understanding the crystalline structure and the ability to smelt one yourself is likely out of scope and probably predefined by domains experts.

Why can’t software work like this?

By the time you've memorized all the cases where a binary tree does and doesn't make sense, you might as well have just learned how they work.

Exactly. I'd be very wary of any professional that hadn't had the intellectual curiosity to at least understand how they work, never mind actually try to implement one.

Where are you working that you don't have tree structures, nor can't recite the generic three lines of pseudocode to traverse it?

I have used trees in almost every moderately complicated program I've ever written, both for fun and at the office.

Sounds a bit like deriving the equations of kinematics over and over again to me. To each his or her own, but that just sounds incredibly boring to me.

I mean, it's literally a for loop:

    def visit(node):
        for child in node.children:
            visit(child)

I don't hire people in my current position, but I certainly think anyone baffled by that would be a giant red flag. It's simpler than FizzBuzz.

I mean, I keep writing for-loops even though I've written many of them before.

Found the imperative programmer.

I'm only half-joking: how often do you need an actual for loop instead of the more abstract concepts of mapping/reducing collections?

I think you made his analogy stronger, not weaker.

As a programmer I am (should be?) concerned with the actual implications of which data structures I use, not how to use them. It should be seamless.

You wouldn't judge an EE on his ability to build wires after you give them some copper and plastic. Sure, he could with tools and a bit of time. It's just a waste of everyone's time and effort.

Some languages don't have first-class functions that you would pass into a map or reduce function, which makes writing loops unavoidable.

But even if you do use map/reduce all the time instead of loops, you're just saying the same thing I did in another way - unless you're doing it for the intellectual thrill of writing a call to map/reduce for the n+1th time.

I wouldn't ask this question but I don't think it's irrelevant.

It shows you that the person understands concepts like recursion, is familiar with data structures like trees, and understands how to express and work with those concepts in code.

Completing this problem should take only a few minutes and tells you whether a candidate has a basic degree of competence. When someone breezes through a problem like this, you learn something. If someone struggles with it, you also learn something. You can use that knowledge to calibrate subsequent questions and find the limits of their knowledge.

I would be skeptical about the effectiveness of software engineer candidate who isn't comfortable with trees or recursion or algorithms like search. It's a weed-out question like FizzBuzz.

I wouldn't ask this question because I think there are better questions available that allows candidates to demonstrate mastery of concepts like these and also have depth so you can go into substantially more detail when the candidate performs well. (And questions that permit multiple solutions, etc.)

Then you want to hire people are good at interviewing; not necessarily people who are good at their job.

I have learned programming on the job, not in school. I think I have a pretty good handle on things like recursion or traversing or implementing structures. But I don't speak the lingo so I don't know what "Implement a recursive depth first traversal of a tree" means exactly. I have been doing this for 25 years now and I don't think I have ever heard anybody formulating a problem that way.

What do you mean, you don't understand "lingo"? Do you not understand english? Because that's an easily understood english sentence, and you just claimed "pretty good handle on things like recursion or traversing or implementing structures".

What's the difficulty with questions like that? But honestly you are lucky if you get asked these questions. Usually it is more about some absurd dynamic programming questions.

I suppose the debate is really between those who want to know and those who don't want to or feel that they don't need to.

That is a perfect example for why you need CS in engineering! Please do a right click view source on this page, what do you see? Look at the comment structure on this page, what do you see? Nobody could work with any of those without using recursive depth first traversal. And once someone understands recursion, the implementation of it is absolutely trivial and obvious. Why would you not use that? The same goes for all the other CS textbook problems, they are specialized screwdrivers that are useless until they aren't.

It's hazing.

I would never hire someone who would rather spend 20 minutes thumbing together a solution that may work instead of someone who would rather read available documentation and literature to find either...

    1. A trusted library that implements a complex feature
    2. A way to abstract away the need for a complex solution
    3. Feel the need to rush an implementation of a mission critical piece of code

I'd say software engineering is more about organization, abstracting, and simplification of a problem than it is about writing complex data structure implementations or complex algorithms. When you do need to implement a complex data structure or algorithm I think it's much more wise to survey available (and current) literature and implement the algorithm after thinking about the problem for a day or two then it is to attempt to implement it yourself in front of 3 people in a stressful time.

I'd expect no one I've ever worked with to be able to correctly meet any business requirement (full battery of tests, an attempt to avoid the more complex solution, documentation for the need and edge cases of an algorithm, real error messages, abstraction or library-extraction of this algorithm into a documented sub-project in our company's git server, etc) I have with a big 5 styled interview question.

It boils down to this:

    1. Your question is so simple it is stupid to ask someone who is actually qualified
    2. Your question is so complicated that anyone who would feel semi-confident in having actually solved it in 40 minutes is someone too dangerous to keep around

> It boils down to this: 1. Your question is so simple it is stupid to ask someone who is actually qualified 2. Your question is so complicated that anyone who would feel semi-confident in having actually solved it in 40 minutes is someone too dangerous to keep around

Really? You don't think there's any possibility of a middle ground here?

I don't think there is anything representative of an employee's quality of work that can be done during a short interview that would have a strong correlate to productivity and quality. I think a real-world take-home problem followed by a meeting-style presentation of your work, your solution, and a Q&A about the implementation with a representative subset of your peers would be better as this would actually be representative of the work load at hand at the company.

the basis of the question its self invalidates middle ground

bingo

> interview questions are engineering problems that either applicable to real world systems, or were inspired from them.

My experience has shown me that in all cases except for two, this is false.

Agreed, a good interview question is also a springboard to a discussion of engineering practices related to the problem.

The only work environment viable engineering answer is to get a library to solve whatever the problem at hand.

At best a top candidate should be able to turn a whitepaper into working code. Knowing trivia rarely translate in solid, productive teams.

Very, very few companies are pushing the what's known barrier.

It's not as simple as being "more of a trade." It's quite similar to the distinction between physics and, say, aerospace engineering in that regard. I'd never describe the latter as being a vocation or trade.

Agreed. Engineering is a "profession". You wouldn't call medicine or law "trades". In some jurisdictions (Canada for example) calling yourself a "software engineer" without having a professional engineering license is unlawful (although lots of people still do it and it's difficult for the regulatory agencies to enforce it at scale).