Identifying Older Androids and iOS devices connecting to Exchange or other services using Sign in Logs and MS Graph
*This post will be followed up by some Powershell queries we can execute to make device counts and version queries easier.
Intro
Part of a rollout strategy for any Mobile Device Management or Mobile Application Management policy roadmap should be what impact is this going to have on users? Your organisation maybe used to working in a certain way with their mobile devices, maybe they are using native apps to sign in to Office365 , i.e Apple Mail or Gmail apps and they may also be using personal devices where currently you have no visibility or asset intelligence of what devices people are using and what apps to connect to Office365.
Scenario
You want to rollout mobile policies to secure your environment, one of the policies you want to turn on is Minimum OS version requirement, currently staff are logging in with their personal and corporate mobile devices, it would be nice to rollout some policies that secure those connections against data loss, but wait! how do we know how many of our users actually use their mobile devices to connect? how do we check if some of the users have got an old Android or Apple iOS that’s going to struggle with the MAM policies. How many people are using Apple Mail or Native Android Mail?
This is where a lot of information can be gleaned from Azure Sign in logs giving you a better picture of user impact before rolling out policies to your organisation.
After playing around with Sign In logs you’ll see that there is no end to the information you can gather and patterns start to emerge that help to explain a lot of what you see when applying Conditional Access policies and how often tokens refresh on your mobile devices.
First thing you’ll notice if you have a large user base is that Sign In logs portal struggles with standard queries you’ll get the message that your queries are being throttled and things grind to a halt, especially with the new feature of Non Interactive Sign Ins this is where MS Graph comes in, to filter our queries and not overload your Azure Tenant with a wide query.
So first lets take a look at the Azure Sign In Logs
Portal.azure.com > Azure Active Directory > Sign In Logs
The First thing you’ll notice is you have some options of
Interactive and Non-Interactive Sign In’s and some Filters you can apply
Briefly Explained
User sign-ins (interactive) — These are as described, where a user logs in on their device and will need to enter credentials.
User sign-ins (non-interactive) — These are sign-ins that happen daily or even every few hours in the background where for example token refreshes are happening.
We are mainly going to focus on non-interactive sign-ins as once a user is logged in to Office 365 using Outlook or Apple Mail its very unlikely they’ll physically sign in again Interactively for months.
So lets take a look.
I’ve gone for non-interactive logins and added a filter iOS in a large environment this will bring back a ton of results, you can also lookup Android and also change the date range from 24 hours to up to a month.
Looking at Application and resource you can add filters to narrow down results further.
Clicking on the results will give you a lot of valuable info
Here’s where we can get some important info
User agent
Note the info is in the User Agent information
Here we can see the user is using Outlook and and iPhone but what OS version? this is where the useful Darwin version table comes in handy
https://www.theiphonewiki.com/wiki/Kernel#iOS.2FiPadOS
So from this we know the user is running iOS 14.7
Microsoft Graph
MS Graph makes looking up Sign in information easier as we can add alot more information to our filters.
So an an example I want to know all the users using iOS 11
If we look at the darwin version table we can see iOS 11 runs on Darwin 17
So if we go to Graph explorer
Graph Explorer — Microsoft Graph
And sign in on the left side
An example query would be
https://graph.microsoft.com/beta/auditLogs/signIns?$top=100&$filter=createdDateTime ge 2022–03–20T06:00:00Z and createdDateTime le 2022–03–25T06:00:00Z and signInEventTypes/any(t: t eq ‘nonInteractiveUser’) and contains(userAgent,’Darwin/17') and contains(userAgent,’iPhone’)
Important note*** Sign in logs only go back 1 month, however in general devices will attempt to refresh their tokens to exchange every 8 hours or 24 hours so you can put in say a 1 week range and get a pretty good idea of the devices out there.
So here we can see a user coming in on an old iOS 11 device, it’ll bring back users details the client they are using etc
Usually I select all and copy this data into excel, from Excel you can filter out all the bloated data and just get say a list of all user email addresses.
Here’s a query for Androids running Android version 8
https://graph.microsoft.com/beta/auditLogs/signIns?$top=100&$filter=createdDateTime ge 2022–03–24T06:00:00Z and createdDateTime le 2022–03–25T06:00:00Z and signInEventTypes/any(t: t eq ‘nonInteractiveUser’) and contains(userAgent,’Android 8')
Also note iPhone doesnt give much away on the hardware its running on, however Androids give away quite alot, here’s some typical outputs from the user agent line.
Final Note
Once you start playing with Sign In logs in Graph there is no end to the info you can retrieve, in another post we will do this with Powershell.
