Hiring for ~10 eng roles listed at astronomer.io/careers, but the focus is specifically on distributed systems, observability, and applied AI.
Astronomer is a primary contributor to Apache Airflow and offers our customers a managed Airflow service with a similar business model as Databricks with Spark or Confluent with Kafka. Our tech stack is primarily Go, Python, and TypeScript. The problems we work on generally center around (1) building out our commercial managed Airflow offering, (2) building a data + infra observability stack on top of Airflow, and (3) building AI products and experiences that are genuinely load-bearing and additive to our customers experience on the platform.
Apply using our website or email me - my email is [firstname]@astronomer.io. Include HN in the subject to make sure I don't miss it! If you're emailing me, please include your resume, GitHub and a short note on what you deem to be the most interesting thing you've worked on.
We’re seeing this all the time - taking traditional workflow orchestration tools and instrumenting LLMs as part of it. It becomes a lot easier to build these because the complexity comes from a) the model, which frontier labs are making easy and b) productionizing a workflow, which workflow orchestration tools make easy. It’s also easy to recognize value because these workflows are often grounded in existing work and thus easy to measure.
We see these patterns do much so that we packaged it up for Airflow (one of the most popular workflow tools)!
Thanks for the feedback. I should probably make it more clear - there is no description field.
The block of text is the 'Likely to outsource' column. I use Perplexity Deep research to try to infer what services the company might need based on it's probable current challenges.
In this case it was: "Cloud infrastructure support to manage rapid scaling, pivoting, and new product integrations as part of business model transformation[3]., Specialized consulting (e.g., industry-specific financial regulations, fintech compliance, and go-to-market strategy) to facilitate entry into new verticals and optimize operational resilience[2]."
Airflow actually uses decorators to indicate something is an explicit task in a data pipeline vs just a utility function, so this follows that pattern!
It also uses an "operator" under the hood (Airflow's term for a pre-built, parameterized task) which can be subclassed and customized if you want to do any customization.
It is _potentially_ more restrictive than writing pure Python functions, but the plus side is that we can interject certain Airflow-specific features into how the agent runs. And this isn't mean for someone who knows agents inside & out / wants the low-level customizability.
The best example of this today is log groups: Airflow lets you log things out as part of a "group" which has some UI abstractions to make it easier. This SDK takes the raw agent tool calls and turns them each into a log group, so you can see a) at a high level what the agent is doing, and b) drill down into a specific tool call to understand what's happening within the tool call.
To your point about the `@task.llm_branch`, the SDK & Pydantic AI (which the SDK uses under the hood) will re-prompt the LLM up to a certain number of attempts if it receives output that isn't the name of a downstream task. So there shouldn't be much finickiness.
reply