Incorrect domain expiration for .EDU domains (EDUCAUSE)

E-werd

This might be two different bugs, but there's a couple things going on here.

Detection of the expiration date is incorrect when looking up our .EDU domains hosted at EDUCAUSE, the sole .EDU registrar.
Despite manually setting the expiration, something is periodically checking and (incorrectly) updating the expiration

For our domain, we're seeing the following info at the bottom of the WHOIS:
Domain record activated: 03-Mar-2008 Domain record last updated: 03-Oct-2023 Domain expires: 31-Jan-2025

What ITFlow is seeing is that 03-Oct-2023, instead of the 31-Jan-2025

wrongecho

Hey,

Domain information is looked up via lookup.itflow.org. This runs periodically via Cron.

Check your domain info here: http://lookup.itflow.org:8080/google.com

E-werd

In an attempt to not dox myself publically, I'm seeing the same result (aside from "domain_name" and "emails" fields) for ours as I see for another college with an edu, SRU.EDU: http://lookup.itflow.org:8080/sru.edu ->

{"address":null,"city":null,"country":null,"creation_date":null,"dnssec":null,"domain_name":"SRU.EDU","emails":["henry.magusiak@sru.edu","andrea.deramo@sru.edu"],"expiration_date":null,"name":null,"name_servers":null,"org":null,"referral_url":null,"registrant_postal_code":null,"registrar":null,"state":null,"status":null,"updated_date":null,"whois_server":null}

Just a whole lot of "null" in that JSON, I'm guessing the difference for EDU domains requires some different querying. I'm not sure where ITFlow is getting 03-Oct-2023 from since it doesn't show at the lookup link.

wrongecho

Thanks for the additional info. I will look into this sometime this week and get back to you 🙂

For reference/if you'd like to dig into this more in the meantime, this is one of the PRs for the whois feature, the python code running that service is here using python-whois. Possibly an upstream issue with that python module.

E-werd

After looking into this a bit, I think a more flexible solution would be to change the python library. I'd suggest this one: https://github.com/DannyCork/python-whois

This library is on pip as "whois" (where the one you're using is "python-whois"). What's great about this one is that it relies on the linux "whois" command which is reliable and robust. You'd have to make sure it's installed and in $PATH.

Here is some sample code I worked up to output from the new lib to the old format: https://pastebin.com/u7T63P9x

I tested a bunch of domains, the output all looked good to me. I'd submit a pull request, but I wasn't able to figure out how to run your code as it sits in the repo.

psiiota

E-werd

That looks like a great python library.

Sadly, ITFlow is written in PHP, so it would be convoluted to implement this. Looking for ways to do this in PHP.

wrongecho

E-werd - I'll look further into this, thanks.

@psiiota - We found it easier to use an external python service for this. There's very few PHP libraries that return who is data in a standard format.

wrongecho

@E-werd

There's an active proposal to archive that library you linked - DannyCork/python-whois322

I've "patched" the existing library we use so that expiry dates for .edu domains come through, would you mind testing please?

wrongecho

My PR for .edu domains was accepted upstream: richardpenman/whois189

This now appears to be resolved.

E-werd

wrongecho Looks good to me, ITFlow is reporting correctly now for me. Thanks!