You can get a surprising amount of information about cognitive processes from eye tracking. I worked on the industrial version of HP Reverb G2 which included a PPG sensor and Tobii eye tracking. Was really an amazing product which suffered from having to use third party platforms like Windows Mixed Reality and OpenXR which didn’t have first class support for what was possible.
One notable takeaway is that eye tracking of saccades is loosely correlated with cognitive load. In fact, “cognitive load” was a product that would be licensed to developers to use in their apps so that difficulty levels could be dynamically adjusted to keep users in a sweet spot for learning.
Especially when combined with a PPG sensor which can in theory detect if a man gets aroused (HP never investigated developing a model for that, but some comments from friends who were DoD-aligned contractors expressed alternating concern or interest in it, because they knew from their own career and independent research what applications might be possible with those technologies)
…it could be used to infer things which someone might prefer to keep private (I.e. which categories of things they have an emotional response to).
Apple definitely is aware of this and is heading off potential issues by not only not providing that data to third party app developers but also even to Apple itself. Apple does have a strong pre-existing framework for dealing with sensitive health data, so they could have decided to capture similar data and treat it the same way they deal with ECG data. HP did not have a HIPAA compatible framework so they threw away 99% of the PPG data and distilled it down to just “heart rate” which was the only data source available to app developers from the PPG sensor.
One notable takeaway is that eye tracking of saccades is loosely correlated with cognitive load. In fact, “cognitive load” was a product that would be licensed to developers to use in their apps so that difficulty levels could be dynamically adjusted to keep users in a sweet spot for learning.
Especially when combined with a PPG sensor which can in theory detect if a man gets aroused (HP never investigated developing a model for that, but some comments from friends who were DoD-aligned contractors expressed alternating concern or interest in it, because they knew from their own career and independent research what applications might be possible with those technologies)
…it could be used to infer things which someone might prefer to keep private (I.e. which categories of things they have an emotional response to).
Apple definitely is aware of this and is heading off potential issues by not only not providing that data to third party app developers but also even to Apple itself. Apple does have a strong pre-existing framework for dealing with sensitive health data, so they could have decided to capture similar data and treat it the same way they deal with ECG data. HP did not have a HIPAA compatible framework so they threw away 99% of the PPG data and distilled it down to just “heart rate” which was the only data source available to app developers from the PPG sensor.