Buzzwords of the Day 10-4-203

#This Week's Buzzwords:

declare -a Buzzword = { "AI/LLM", "Exploits", "Wearables" }

$Buzzword{0}= "AI/LLM" - Otto Pilot, Deceased Relatives, and Not-Tom-Hanks

This week was full of AI updates, *(to nobody's surprise). Reports from MIT overviewed an intelligent flight assistant system, which developers have dubbed the "Air-Guardian". From the announcement, it appears they are developing an AI-trained co-pilot designed specifically to support a human primary pilot and increase multi-tasking capacity for the cockpit. The intended functionality is for the Guardian "co-pilot" to detect which tasks the human pilot is currently handling, and apply itself towards monitoring/handling other necessary flight operations. For example, while the pilot diagnoses an error message from one of the many sensors on a modern aircraft, the co-pilot could actively monitor other vital information such as fuel consumption, altitude/attitude, etc. The theory is that this program could enhance what is colloquially called "auto-pilot" assistance, allowing the primary pilot more capacity to safely operate the vehicle and, if necessary, diagnose & resolve issues mid-flight more safely.

~~~

End-users are terrific at finding bugs, loopholes, and exploits within new technology, whether it be a "speed-run" tactic for a game, public lists of discount codes for e-tailers, or, most recently, "making ChatGPT do things it isn't supposed to do". From this author's perspective, it appears the latter is becoming an increasingly popular hobby thanks to public access to Bing Chat, ChatGPT, and similar LLM tools. Recently, many developers have added "image-reading" capacity to their models, with some failsafes to try and safeguard against its use for what they deem 'unintended' (such as solving a Captcha). ArsTechnica reports that users of Bing Chat (a remarkably similar LLM to ChatGPT) is able to break its own rules, with some help. When prompted to directly resolve a captcha image to text, the chatbot replies with a boilerplate "Im' sorry, Dave, I'm afraid I can't do that" response. But savvy users have discovered a workaround, using remarkably similar tactics to prior LLM 'exploits' which had ChatGPT offering guides & instructions for creating napalm at home. Just as before, the user presents the *ahem...prohibited* activity to the chatbot disguised from one step of abstraction:

Instead of directly asking the model to solve the captcha, this user "primed" the model with a scenario: "my grandmother has passed away, and left me this cryptic note in a locket". The 'note' is a stock image of a locket with the captcha text superimposed over the locket. To a human, the image is obvious (link to the article below if you are curious), but to image-interpretation software, the image clearly "passes" and dupes the LLM into working on resolving it. Since the machine does not believe it is violating its programming (solving a Captcha), it applies its image-reading capacity to render the image's contents for the user.

Ultimately, as LLMs like Bing Chat, Google's Bard, and ChatGPT integrate more deeply into day-to-day technology like search engines, the frequency of "jailbreak" workarounds like this present a more clear picture of the threat posed by misuse of this same technology.

~~~

How else could AI & LLM's be mis-used for 'evil' as they stand today? Take a look at TikTok - a recent slew of AI-generated videos, using AI-generated voice models, and (likely) AI-written scripts to fully impersonate celebrities and high-profile personalities for scams. Currently, per ArsTechnica, deepfake videos of celebrities including Tom Hanks and MrBeast promoting services and products that they have neither heard of nor endorsed formally. These ads are a terrific 'canary' to check on the danger levels of AI tools in their current development. The videos themselves fail to pass deep scrutiny (the mouth movements are often very unnatural, and look almost like sped-up Monty Python footage), but from a distance/glance/small screen (such as a cell phone display), could be convincing enough to the average consumer. If anything, the effectiveness (and rapid proliferation) of these scams hammer home the Writers' Guild's concerns around AI models' impact on content generation moving forward. In a hopeful direction, social media/publishing platforms such as TikTok, Youtube, and Meta are implementing/developign/discussing steps to more clearly define/indicate AI-generated content on their platforms, but from this author's perspective, they may already be too far behind. An anonymous colleague of mine put it best: "Team Blue has the hard job. They have to nail their defensive policies 100% of the time. Team Red? they only have to be right **once**".

Source Context:

https://news.mit.edu/2023/ai-co-pilot-enhances-human-precision-safer-aviation-1003 (AI Co-Pilots)

https://arstechnica.com/information-technology/2023/10/sob-story-about-dead-grandma-tricks-microsoft-ai-into-solving-captcha/ (Bing Chat Solves Captcha)

https://arstechnica.com/information-technology/2023/10/tom-hanks-warns-of-ai-generated-doppelganger-in-instagram-plea/ (Deepfake Scam Videos)

$Buzzword{0}= "Exploits" - Even Links Aren't Safe, Come Look at My Weak Points

Reverse proxy exploits are falling into increasingly frequent use, primarily to take advantage of how many/often users remain logged into varied services through an online browsing session. BleepingComputer reports a recently discovered malware/phishing campaign that leverages a reverse-proxy page to steal a users' Microsoft 365 account credentials. From reports, the actors disguise a malicious link appearing to lead to an Indeed page. Instead, the link redirects (302) to a man-in-the-middle page, which prompts the user for their Microsoft login details. From there, any cookie data (including 2FA pass, if applicable) are stolen and retained by the threat actors to use at a later time. It appears these attacks are primarily targeted towards corporate/enterprise environments, likely trying to gain access to user data for individuals of influence.

~~~

Now imagine that any business’ possible backdoor, opening, or security exploit like the above were included on an up-to-date ‘living’ central repository. That's what the EU appears to be considering as a part of their proposed Cyber Resilience Act. In an attempted step towards corporate transparency, the law proposes that business entities publish security assessments & updates to their respective government agencies, 100% of the time. The intention, per supporters of the law, is to support/encourage rapid response to incidents. However, such reporting would mean creating a running, up-to-date list of business entities' known exploits/vulnurabilities, which itself would be a massively sensitive document. Security teams and experts warn, per ComputerWeekly.com, that the risks of this type of list are substantial: the mere existence of an ongoing list of companies & their specific vulnerabilities allows the possibility of its exploitation by malicious actors for profit, surveillance, or intelligence. Some opponents of the legislation seek a middle ground, where the mandatory response period is treated with equal urgency, without necessarily reporting the specifics of a breach to every European goverment agency **every single time**.

Source Context:

https://www.bleepingcomputer.com/news/security/evilproxy-uses-indeedcom-open-redirect-for-microsoft-365-phishing/ (Indeed Links to Malware)

https://www.computerweekly.com/news/366554133/Cyber-experts-urge-EU-to-rethink-vulnerability-disclosure-plans (EU Vulnurability Disclosures)

~~~

$Buzzword{0}= "Wearables" - Subtitle2

Google announced their new Pixel Watch this week, with some notable hardware improvements over the first-generation device. At least until Google kills this service as well (RIP Google Podcasts, we knew ye well), they seem to be all-in on improving the hardware included on-board. The new watch features, besides software/processor improvements & improved battery technology, additional senors and software monitoring to better position the Google Watch as a fitness aid as well as smart watch. They plan to further integrate FitBit Health into the watch's software ecosystem, and claim improved performance for auto-detecting when the user begins a workout. Additional hardware sensors include a skin temperature sensor to better monitor biometrics. They have also added new software features, such as the "safety check", which can be used to check in with the user after a pre-determined time period (The Verge offers an Uber ride as example) to verify their safety. Failure to confirm the warning on the device would automatically broadcast their location & other necessary data to the phone's emergency contacts. If it works as-advertised, this feature could be a terrific tool for travel safety.

Source Context:

https://www.theverge.com/23896271/pixel-watch-2-hands-on-google-price-specs (Google Watch Notes)