Run DNS and Mailbox Checks in Airflow DAG Before Email Send
Prevent bounces and protect sender reputation by validating emails with DNS and mailbox checks in your Airflow DAG before sending. Automate list hygiene.
Why skipping DNS and mailbox checks before email sends leads to wasted effort
You sent an email campaign. The dashboard says 95% delivered. But open rates are near zero. You’re not wrong—your list has silent drag. Invalid addresses, role accounts, disposable domains: they don’t engage, but they still count.
Sending without pre-verification is like launching a delivery drone into a storm. It’s not just the failed delivery—it’s the damage to your reputation, the lost time, the inflated costs. A single hard bounce can hurt your sender score. DNS and mailbox checks in your Airflow DAG prevent this before the first send.
Running DNS and mailbox checks in Airflow DAG before email send isn’t a luxury. It’s a fix for the invisible friction in your pipeline: bounce fatigue, reputation damage, wasted bandwidth.
Key takeaways
- Skipping pre-send validation leads to hard bounces that degrade sender reputation over time.
- DNS and mailbox checks in Airflow catch invalid, role-based, and disposable emails before they hit mailing servers.
- Preventing errors in the pipeline is faster and cheaper than cleaning up failed sends post-delivery.
What DNS and mailbox checks reveal about an email address
Before sending an email, DNS and mailbox checks expose whether an address is technically valid and actively receiving messages. DNS checks confirm the domain exists and has proper infrastructure—like MX records for routing and SPF for sender authorization. Mailbox checks go further by contacting the server to verify if the specific inbox accepts mail. Together, they catch formatting errors, non-existent domains, closed accounts, or blocked addresses—preventing bounces, damaging sender reputation, and wasted sends. This dual-layer validation ensures only deliverable addresses progress to your email campaign.
DNS checks: Verify the domain's foundation
When you run a DNS check, you’re looking at the domain’s public records. An MX record must exist to route mail to the correct server. SPF records define which servers are allowed to send on the domain’s behalf—critical for authenticating your sending source. TXT records may hold additional policies, like DMARC, which helps prevent spoofing. Without these, even a correctly spelled email address can’t be delivered. This step catches domains that don’t exist, have no mail routing, or lack basic sending policies.
Mailbox checks: Confirm inbox readiness
Once the domain is verified, mailbox checks probe the receiving server directly. This isn’t just guessing—it’s simulating an actual email send to see if the server accepts or rejects the address. Responses like "user unknown" or "mailbox full" reveal real delivery issues. This step detects closed accounts, role-based addresses (like support@ or admin@), or disposable email domains. These are common sources of hard bounces and can hurt sender reputation if not filtered early.
Running both checks in an Airflow DAG before every send ensures your email list stays clean. Tools like bulk email list cleaning automate this process at scale, integrating seamlessly into workflows. For real-time verification within pipelines, our API handles validation in milliseconds. These checks are industry-standard practices—RFC 5321 outlines SMTP behavior, and organizations like Spamhaus track patterns associated with invalid or harmful addresses. The goal isn’t perfection, but meaningful risk reduction. Every failed delivery harms deliverability; catching them before sending avoids the damage.
How to integrate DNS and mailbox validation into your Airflow DAG
You can run DNS and mailbox checks in your Airflow DAG by writing a custom Python operator that calls the Email List Validation API. This checks each email for syntax, domain validity, mailbox existence, and risk flags before sending. It reduces bounces and protects sender reputation, which is critical for inbox placement. Use the real-time API to check addresses one by one or in small batches, then filter out invalid or risky emails before triggering your send task.
Set up the custom operator
- Define a custom Airflow PythonOperator that wraps the Email List Validation API call. This allows you to inject validation logic directly into your DAG flow. Use the real-time email verification API to validate individual addresses with full DNS and mailbox checks.
- Write a function to process the email list in small batches—50 to 100 emails per request—to stay within rate limits and avoid timeouts. The API returns verdicts like
valid,invalid,catch-all, orrisky. Handle HTTP errors and retry logic for flaky connections. - Parse the API response and categorize each email.
invalidmeans the address fails syntax or DNS checks.riskyindicates a possible disposable, role-based, or high-bounce risk (e.g.,admin@...ortempmail.com). Use RFC 5321 and RFC 5322 as reference for understanding how mail servers process addresses. - Filter out
invalidandriskyemails before proceeding to the send task. Keepvalidandcatch-alladdresses, but mark them for separate tracking. Catch-alls are known to be soft-bounced during delivery, so treat them as lower-priority. - Pass the cleaned list to the downstream send task. This ensures only valid, deliverable emails are transmitted—reducing bounces and protecting your sender reputation. Monitoring bounce rates helps validate your filtering logic over time.
Validate in production with confidence
Run this validation step daily or before every send campaign. Keep logs of the response codes and filter outcomes for audit and debugging. Use the API’s bulk capability — bulk email list cleaning — to pre-process large datasets when you’re not running real-time workflows.
Remember: Even with validation, sender reputation and content quality still affect inbox placement. Tools like Spamhaus and MxToolbox can help monitor blocklist status, but they don’t replace pre-send verification. Keep your list clean, your sends targeted, and your reputation intact.
What each verification verdict means in practice
You don’t send to “valid” addresses without question, but you also don’t ignore “catch-all” or “risky” results. Valid means the address exists and the mailbox is responsive — likely deliverable. Invalid means it’s dead: wrong format, fake domain, or failed test send. Don’t send. Catch-all domains accept any email — common in spam traps or shared inboxes. Sending to them risks being flagged as spam. Risky means the address passed DNS but failed mailbox tests or is known to bounce. Treat with caution. Let’s break down what each one actually means when it hits your Airflow DAG.
Verdicts and what to do next
- Valid: The mailbox exists and responds to test deliveries. It passed DNS (MX, SPF, DKIM) and SMTP handshake. Your email is likely to reach the inbox. You can proceed with confidence.
- Invalid: Format error (like missing @), non-existent domain, or hard bounce during verification. These should be filtered out before any send. Sending to them causes delivery failures and hurts sender reputation. RFC 5321 defines how mail servers reject invalid addresses.
- Catch-all: The domain accepts all addresses, regardless of existence. This is common in older systems or spam traps. Sending here increases risk of being marked as spam. Many spam filters now flag such domains. Avoid sending to them unless you have strong intent and tracking in place.
- Risky: The address passed DNS but failed inbox tests — high bounce rate, suspected disposable, or known to trigger filters. You might send, but only after monitoring and with a clear fallback. These often come from list brokers or scraped sources.
Integrating checks into your Airflow DAG
Run DNS and mailbox checks as a pre-send task in your DAG. Treat each verdict as a decision rule:
- Valid → Proceed to send.
- Invalid → Remove or flag for review.
- Catch-all → Mark for suppression or skip unless you’re doing targeted outreach.
- Risky → Queue for approval, send at lower volume, or test inbox placement first.
Use real-time verification to catch edge cases before batch sends. For bulk cleaning at scale, tools like bulk email list cleaning help weed out invalid or risky addresses upfront. You can also integrate the verification API directly into your DAG for instant checks on new entries. Consistent filtering reduces bounce rates and protects your sender reputation. A clean list isn’t just efficient — it’s a requirement for long-term deliverability.
Integrate Email List Validation with SendGrid in Airflow
You can run DNS and mailbox checks in Airflow DAG before email send by using the SendGrid hook to connect to your account, adding a validation task that calls the Email List Validation API, filtering invalid and risky addresses from your list, then passing only clean recipients to the SendGrid operator. This reduces bounces, improves sender reputation, and boosts inbox placement.
Set up the SendGrid connection
Start by configuring the SendGrid connection in Airflow using the SendGrid hook. This sets up authenticated access to your SendGrid account, allowing your DAG to send emails after validation.
Ensure the API key has permissions to send emails. Use Airflow’s Connections UI or config files to store the key securely. Misconfigured credentials will fail the send task before any validation even runs.
- Import and configure the
SendGridHookin your DAG. - Add your SendGrid API key as a connection in Airflow’s Connections interface, using the
sendgrid_defaultconnection ID. - Verify the hook works by testing it with a simple task before chaining in validation.
Run validation and clean the list
Next, insert a validation task that calls the Email List Validation API. This checks DNS records, mailbox existence, and risk signals like disposable domains or role accounts.
Use the API response to filter the list dynamically before sending. Only addresses marked as valid or clean-risk should proceed.
- Add a PythonOperator task that runs right before the SendGrid operator.
- Use the Email List Validation real-time API to verify each email in your list.
- Parse the response and remove any entries with verdicts like invalid, catch-all, disposable, or risky.
- Pass only valid addresses to the SendGrid operator using the
toparameter in the SendGridOperator task.
By doing this, you ensure only deliverable, trustworthy addresses are sent. You avoid hard bounces, reduce strain on your sender reputation, and improve inbox placement. According to industry standards, a bounce rate above 2% can trigger blacklisting — this process keeps it below that threshold.
This workflow follows established email best practices, including proper DNS configuration and mailbox verification. You’re not just sending emails; you’re ensuring they land where they should — and stay there.
For larger lists, consider using bulk email list cleaning at https://emaillistvalidation.com/bulk-email-list-cleaning to process high-volume verification efficiently.
Real-time verification API: How it works under the hood
You can run DNS and mailbox checks in Airflow DAG before email send by using a real-time verification API that performs a full SMTP handshake with the target mail server. It checks MX records to confirm valid mail routing, connects to the receiving server, and simulates sending a test message — without delivering any content. The entire process completes in under one second per address, enabling high-throughput validation at scale.
SMTP handshake: The core of real-time validation
When you submit an email address, the API initiates a real SMTP session — not just a DNS lookup. It resolves the domain’s MX records to find the correct mail server, then attempts a connection. This proves the domain is configured for email delivery, not just a static DNS entry. If the server rejects the connection, the email is invalid or blocked.
Let’s say you're sending transactional emails through an Airflow DAG. Skipping this phase risks sending to non-existent mailboxes or services that reject connections outright. Tools like the real-time verification API catch those failures early, reducing bounces and protecting sender reputation.
Testing mailbox acceptance without sending content
After establishing the SMTP connection, the API sends a MAIL FROM and RCPT TO command to test whether the mailbox accepts incoming mail. It doesn’t send a message body — it’s a dry run. If the server responds with a 250 OK, the mailbox likely exists and is accepting messages. If it returns a 550 or 553, the address is invalid or quarantined.
This method mirrors what actual email servers do during delivery. According to RFC 5321, the SMTP protocol specifies that RCPT TO commands are used to verify recipient existence. By following the standard, the API avoids false positives — you’re not just checking if the domain is valid, but whether the specific mailbox can receive mail.
Results are returned in under 1 second — fast enough to integrate into Airflow DAGs that process thousands of emails. The API returns clear verdicts: valid, invalid, catch-all, or risky — each based on real server behavior, not heuristics or guesswork. This accuracy lets you filter out problem addresses before they hit your ESP, improving inbox placement and reducing the risk of being flagged as spam.
Why relying solely on format validation is not enough
You can’t assume an email is deliverable just because it looks right. A valid format like [email protected] passes basic syntax checks, but that doesn’t mean the domain exists, accepts mail, or that the inbox is active. Without probing DNS records and mailbox availability in real time, you risk sending messages to addresses that will bounce, be delayed, or never land in an inbox.
DNS and mailbox checks uncover what format validation misses
Just because a domain has MX records doesn't mean it accepts incoming messages. Some domains use greylisting, rate limiting, or require authentication that blocks unapproved senders—even if the address format is correct. These behaviors aren’t detectable by parsing rules alone. A mailbox might be technically “valid” but inactive for months, or even completely shut down.
Mailbox checks confirm whether an inbox is currently accepting mail. They test the actual SMTP handshake with the target server, simulating a real delivery attempt. This is how you distinguish between a valid format and a valid, active recipient. Without this step, your list remains a high-risk guessing game.
Greylisting, rate limits, and role accounts distort deliverability
Even with a working domain, services like SMTP greylisting temporarily reject mail from unknown senders. This means an address may fail today but succeed tomorrow. Format validation won’t detect this. Likewise, some domains enforce strict inbound policies that block mail from third-party sources—no matter how clean the syntax.
Role accounts like admin@ or marketing@ may have valid formats but are often monitored, auto-deleted, or redirected. These aren’t real people. They’re not responsive, and their inclusion hurts sender reputation over time. Real-time checks help filter out these dead ends before they ever hit your send queue.
You can’t rely on static rules to keep up with dynamic server behavior. DNS and mailbox probing give you a live read on deliverability—no assumptions, no guesswork. The same checks you’d run before sending in Airflow DAGs can be automated at scale to prevent wasted sends.
Learn how to validate email addresses before sending with our real-time verification API, or clean entire lists with bulk verification. These tools go beyond syntax to examine active inbox status and server behavior, giving you a realistic picture of deliverability. For deeper insight, test inbox placement with our inbox placement tool.
How to reduce bounce rates using Airflow and Email List Validation
You can cut bounce rates by up to 90% by validating every email in your list before sending using a pre-send DAG task in Airflow. This process checks DNS records, mailbox health, and domain reputation in bulk. You’ll catch invalid addresses, catch-all domains, and disposable emails—common sources of hard bounces and spam trap hits—before they hurt deliverability. The results are logged for audit trails and compliance, and the system scales to clean tens of thousands of emails across multiple domains.
Implement your validation pipeline in Airflow
- Use a pre-send DAG task to run bulk validation on your entire email list before any send.
- Call the real-time verification API from within your DAG to process thousands of emails in parallel with low latency.
- Check DNS records (A, MX, SPF, DKIM) and mailbox existence to confirm real, active addresses.
- Filter out catch-all domains—those that accept any email—because they’re commonly used by spammers and can expose you to spam traps.
- Block disposable email domains (like tempmail.org or 10minutemail.com) which are rarely used for legitimate engagement and increase bounce risk.
- Use the bulk verification service for large-scale cleanups when you need to process lists of 100k+ addresses.
Track and audit every decision
- Store validation results in a structured log—include verdicts like “valid,” “invalid,” “catch-all,” or “risky”—with timestamps and reasons.
- Use the logs to prove compliance during audits or to debug deliverability issues.
- Review trends: if a high number of “risky” emails come from a certain domain, investigate your acquisition source.
- Integrate with tools like Mailchimp or Klaviyo via the native integrations to keep your campaigns clean at the source.
- Monitor sender reputation: sending to invalid or disposable domains harms your sender score over time—this check helps preserve it.
Validating your list is not a one-time cleanup. It’s a systematic, repeatable step that keeps inbox placement steady and reduces the risk of being blacklisted.
For more on email deliverability best practices, see the SMTP RFC or industry guidance from Return Path, which shows that list hygiene correlates directly with open and conversion rates. You’re not just avoiding bounces—you’re improving trust, performance, and reach.
What to do with high-risk or catch-all addresses
If your Airflow DAG flags an address as high-risk or catch-all, don’t send to it in bulk. Mark it for manual review or restrict it to low-volume, non-promotional campaigns. Sending to these addresses harms your sender reputation, increases bounce rates, and can trigger ISP filters. Use the insights from DNS and mailbox checks to refine your list hygiene—proactively remove domains or patterns linked to risky addresses. You can use the in-app AI assistant to surface domain-level trends behind these verdicts, helping you clean at scale.
Why avoid mass sends to catch-all or high-risk addresses
Even if a catch-all address accepts your email, it's not a real user. Sending to such addresses signals poor list quality to inbox providers. A single high-risk address across hundreds of thousands of emails can degrade your sender reputation, especially if it triggers bounces or engagement drops. ISPs like Microsoft and Gmail use reputation signals—including hard bounces and low engagement—to assess legitimacy. A consistent pattern of sending to invalid or risky addresses, even in small volumes, can eventually result in filtering or throttling.
Let’s be clear: a catch-all does not mean “valid.” It means “accepts anything.” That’s often a red flag for bots or disposable accounts. According to Return Path data, domains with high catch-all ratios correlate with elevated spam complaints and low engagement—common signals that lead to delivery issues.
Improve your list-cleaning strategy
Instead of treating each flagged address as an isolated case, look for patterns. Are entire domains or email patterns (like team@company, admin@, or user123@) showing up consistently as high-risk? If so, update your filtering rules in your Airflow DAG to block those domains or match those formats at ingestion. This prevents future contamination.
Use the in-app AI assistant to analyze clusters of risky emails. It will help you identify whether the risk stems from a specific domain, a common pattern, or a known disposable provider. This visibility lets you act proactively—removing entire domains before they’re even tested, rather than cleaning after the fact.
For teams using Airflow, this means building checks into your DAG that validate email structure, domain MX configuration, and mailbox behavior—then using that data to filter at runtime. You can run real-time validations via the real-time email verification API or perform bulk cleanups with the bulk email list cleaning tool. Both integrate smoothly with your existing workflows.
Email List Validation’s accuracy: what it means for your Airflow pipeline
With 98.9% accuracy, Email List Validation’s real-time API cleanly separates valid emails from invalid ones before your Airflow DAG triggers a send. This reduces false positives, cuts bounce rates, and preserves sender reputation—keeping your messages in inboxes, not spam traps. It’s not just about filtering out junk; it’s about trusting your pipeline with high-fidelity data.
High accuracy means fewer missed opportunities
When you run a full list validation in your Airflow workflow, 98.9% accuracy means only about 1.1% of addresses are misclassified. That’s a meaningful reduction in false negatives—real users aren’t lost due to a misjudged syntax error or a temporarily down mailbox. You send only to addresses that are not just syntactically correct but also likely to receive messages.
For example, a catch-all domain might pass basic syntax checks but fail real-world delivery. Our API identifies those by probing the mailbox behavior—checking if mail is accepted or rejected at the server level. That’s not just a syntax check; it’s active verification. As defined in RFC 5321, SMTP transactions are the gold standard for determining deliverability readiness.
Cost efficiency built into the design
You’re not paying for unused verification credits. With Email List Validation, purchased credits never expire. This matters in Airflow workflows where you might validate lists at different intervals—weekly, monthly, or after list growth events. There’s no pressure to burn through a quota before it expires, so your infrastructure costs stay predictable and efficient.
A high-accuracy solution means less cleanup after sends. No need to scrub your list manually or retrain models on false negatives. Over time, this leads to consistent inbox placement. According to Spamhaus, consistent sending patterns from clean lists correlate directly with better deliverability—fewer bounces, fewer blocks.
Let’s say you’re using Airflow to trigger monthly newsletters. Without validation, your pipeline might send to 200 invalid addresses. With the API, you catch those before the send, avoiding reputation hits. You don’t just save bandwidth—you protect your sending domain.
For teams using Airflow for scheduled sends, integrating a high-accuracy verification step isn’t optional. It’s foundational. And with real-time email verification via API, your pipeline stays fast, reliable, and aligned with industry standards for mail hygiene.
Conclusion: Prevent problems before they happen in your email workflow
Running DNS and mailbox checks in your Airflow DAG before sending emails eliminates invalid addresses early, reducing bounce rates and protecting your sender reputation.
Automating verification with the Email List Validation API integrates seamlessly into your pipeline, ensuring every send is clean, reliable, and scalable.
Sources
- Segmented campaigns also protect list health, driving 9.37% fewer unsubscribes, 4.65% fewer bounces, and 3.90% fewer abuse reports than unsegmented sends. — Mailchimp (2025)
- GetResponse benchmarks put the average unsubscribe rate at 0.15% and the average spam complaint rate below 0.01% of sends. — GetResponse Email Marketing Benchmarks (2024)
Keep reading
- Engagement, segmentation and campaign benchmarks (complete guide)
- Detect Changes in Email Provider Response Rates Between Two Runs
- What Does Domain Mismatch Mean in Email DSN Logs?
- Enhance Email Campaign Performance by Removing Temporary Domains
- DSN 5.7.1 Email Rejection Causes in Microsoft Exchange 2024
Ready to put this into practice? Email List Validation verifies emails with 98.9% accuracy — start with 100 free verifications.
Frequently asked questions
Can I run DNS and mailbox checks in Airflow without third-party tools?
Yes, but writing and maintaining an SMTP client with full DNS resolution, greylisting avoidance, and server response parsing is complex. Using a dedicated API like Email List Validation is more reliable and faster.
How many emails can I validate in a single Airflow DAG run?
The Email List Validation API supports bulk processing of thousands of addresses in a single request, making it suitable for large-scale campaigns.
Does this approach work with role-based addresses like support@ or info@?
Role-based addresses often return catch-all or risky verdicts. They are common in spam traps—filter them out to maintain sender reputation.
What happens if an email is flagged as 'catch-all'?
Catch-all domains accept all incoming mail, increasing the risk of being flagged as spam. These should be excluded from mass campaigns.
Can I integrate this verification with Mailchimp or Klaviyo?
Yes—Email List Validation integrates with Mailchimp, HubSpot, Klaviyo, and SendGrid. Run verification in Airflow, then sync clean lists to these platforms.
Is the API suitable for real-time email checks during sign-up?
Yes. The real-time verification API is optimized for low-latency responses, making it ideal for validating user emails at signup.
How do disposable email domains affect deliverability?
Disposable domains are often used for spam or fake accounts. Sending to them harms sender reputation and increases bounce rates.
Do I need to handle API rate limits in Airflow?
The Email List Validation API manages rate limiting internally. Use batch requests to maximize throughput without manual throttling.
Can I use the validation results to segment my email list?
Yes. Use verdicts like valid, risky, catch-all, or invalid to create targeted segments and improve engagement.
How does DNS checking prevent false positives?
DNS checks confirm the domain exists and has valid MX records. Addresses on non-existent or misconfigured domains are rejected early.
What’s the best way to start validating emails in Airflow?
Begin with 100 free verifications in the Email List Validation dashboard, then integrate the API into your DAG using Python.
Does Email List Validation support GDPR compliance?
Yes—Email List Validation provides audit logs and data transparency. You control the data and can delete records at any time.