feed.xml

<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.1.0">Jekyll</generator><link href="https://mattcasmith.net/feed.xml" rel="self" type="application/atom+xml" /><link href="https://mattcasmith.net/" rel="alternate" type="text/html" /><updated>2024-07-20T15:31:47+01:00</updated><id>https://mattcasmith.net/feed.xml</id><title type="html">MattCASmith</title><subtitle>A blog about cyber security and technology</subtitle><entry><title type="html">Endpoint detection and response (EDR) - setting the record straight</title><link href="https://mattcasmith.net/2024/07/20/crowdstrike-bug-edr-setting-record-straight" rel="alternate" type="text/html" title="Endpoint detection and response (EDR) - setting the record straight" /><published>2024-07-20T01:00:00+01:00</published><updated>2024-07-20T01:00:00+01:00</updated><id>https://mattcasmith.net/2024/07/20/crowdstrike-bug-edr-setting-record-straight</id><content type="html" xml:base="https://mattcasmith.net/2024/07/20/crowdstrike-bug-edr-setting-record-straight">&lt;p&gt;When I went to bed on the evening of Friday 19th July, I couldn’t sleep. It was a stuffy summer’s night in London, and the adrenaline was still pumping through my veins after one of the more notable days in recent memory for cyber security. Still, laying awake gave me time to reflect on what I’d seen.&lt;/p&gt;

&lt;p&gt;Cyber security is a function that normally runs quietly in the background, so when it does make the headlines, it’s often for the wrong reasons. In the past, we’ve seen huge incidents like &lt;a href=&quot;https://www.wired.com/story/notpetya-cyberattack-ukraine-russia-code-crashed-the-world/&quot; target=&quot;_blank&quot;&gt;NotPetya&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/WannaCry_ransomware_attack&quot; target=&quot;_blank&quot;&gt;Wannacry&lt;/a&gt; become global news, but yesterday, it was a security product itself that stole the show.&lt;/p&gt;

&lt;h3 id=&quot;what-exactly-happened&quot;&gt;What exactly happened?&lt;/h3&gt;

&lt;p&gt;In short: CrowdStrike pushed a bad update to Falcon, a leading endpoint detection and response (EDR) product, that caused a lot of Windows computers to fail to boot. This triggered chaos due to the number of computers around the world running the software. A workaround was quickly discovered, but it needed to be completed manually on each affected system, causing a huge headache for IT administrators.&lt;/p&gt;

&lt;p&gt;President and CEO George Kurtz &lt;a href=&quot;https://x.com/George_Kurtz/status/1814235001745027317&quot; target=&quot;_blank&quot;&gt;took to X&lt;/a&gt; to try to reassure the world that no cyber security incident had occurred, and said that CrowdStrike had deployed a fix. He would later share more apologetic posts, including &lt;a href=&quot;https://www.crowdstrike.com/blog/our-statement-on-todays-outage/&quot; target=&quot;_blank&quot;&gt;a letter on the firm’s website&lt;/a&gt;, but CrowdStrike as a company is not the focus of this post.&lt;/p&gt;

&lt;p style=&quot;width:100% !important; text-align: center !important&quot;&gt;&lt;img src=&quot;/wp-content/uploads/2024/07/crowdstrike-edr_00.png&quot; class=&quot;half&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Cyber security is a cat and mouse game between those who wish to do harm and those who protect our systems, and for this reason security software needs constant updates to detect and prevent the latest attacker techniques. To effectively monitor endpoints, the software must also run in a privileged state, which means that if it fails, it can easily cause more problems than standard user mode applications.&lt;/p&gt;

&lt;p&gt;Therefore, although CrowdStrike will certainly want to go over its internal testing and review processes, it’s important to remember that a similar thing could happen if any other security vendor made a similar mistake (for example, we saw &lt;a href=&quot;https://www.theregister.com/2019/10/15/blue_screen_of_death_symantec/&quot; target=&quot;_blank&quot;&gt;a similar incident on a smaller scale&lt;/a&gt; with Symantec in 2019).&lt;/p&gt;

&lt;h3 id=&quot;misconceptions-and-inaccuracies&quot;&gt;Misconceptions and inaccuracies&lt;/h3&gt;

&lt;p&gt;There are a thousand articles out there about the bug itself, how it was fixed, and what this means for CrowdStrike, so I won’t retread that ground here. After the initial shock of what was happening, the next thing that caught my attention was the way CrowdStrike - and by extension EDR as a whole - was being discussed, which showed a complete lack of understanding of the technology and the entire sector.&lt;/p&gt;

&lt;p&gt;The internet is a terrible place to read about things you’re an expert in, but this trend wasn’t limited to the general population on social media. Even on relatively technical forums like &lt;a href=&quot;https://news.ycombinator.com/&quot; target=&quot;_blank&quot;&gt;Hacker News&lt;/a&gt; there was a strange tone to the conversations, and experienced technology journalists at respected publications dropped huge inaccuracies into their writing. Broadly, these misconceptions fell into three categories:&lt;/p&gt;

&lt;p&gt;      1. “EDR is anti-virus software”&lt;br /&gt;
      2. “EDR is corporate spyware”&lt;br /&gt;
      3. “EDR isn’t worth the risk”&lt;/p&gt;

&lt;p&gt;I don’t think any of these inaccuracies were spread maliciously. I think they came more from a lack of understanding of what an EDR tool is and does and what cyber security teams are there to do. Therefore, I thought it would be worth writing an article covering the truth behind each of these sentiments.&lt;/p&gt;

&lt;h3 id=&quot;1-edr-is-anti-virus-software&quot;&gt;1. “EDR is anti-virus software”&lt;/h3&gt;

&lt;p&gt;Since home and office computing became mainstream around the 1990s, the vague security threat of “viruses” has loomed. In that era, some of these were actually quite fun. You wouldn’t want them to spread to your PC, but they’d often do quirky things like &lt;a href=&quot;https://www.youtube.com/watch?v=z7g-v3d7-Gk&quot; target=&quot;_blank&quot;&gt;make all the letters fall to the bottom of your screen&lt;/a&gt; or challenge you to a game of cards to save the contents of your hard drive.&lt;/p&gt;

&lt;p&gt;As malware became more serious, the internet and portable media allowed it to spread more freely, and awareness grew. It became common practice to buy an anti-virus product to identify and stop malicious software, and therefore it was only natural that the name “anti-virus” became a casual synonym for all security software, much like any video game console was sometimes referred to as “a Nintendo”.&lt;/p&gt;

&lt;p&gt;But the truth is that traditional anti-virus software is quite limited. It relies on hashes and signatures to detect malware, and therefore if the author finds a way to alter their malicious code slightly each time it spreads (polymorphism), it can evade detection. Because it scans for files, standard anti-virus software can also miss attackers’ abuse of legitimate tools, because the files on disk are all benign.&lt;/p&gt;

&lt;p&gt;The other limitation of traditional anti-virus software is that it usually only provides a small set of automated remediation actions. If it detects malware, it will at most usually terminate any associated processes and quarantine the files. Any further action requires manual access to the computer.&lt;/p&gt;

&lt;p&gt;EDR tools, including CrowdStrike, do a whole lot more than this. While they &lt;em&gt;do&lt;/em&gt; scan files and quarantine them (CrowdStrike even has a component called next-gen anti-virus, or NGAV), to call them “anti-virus” is to sell their capabilities seriously short. EDR products essentially monitor everything that happens on a computer and pick through it with a fine-toothed comb for malicious behaviour - even for threats that do not involve files, like in-memory malware or attackers with direct hands-on-keyboard access.&lt;/p&gt;

&lt;p&gt;When malicious activity is detected, EDR also gives cyber security professionals many more options to respond to attacks. If it looks like malware could spread between systems, they can disconnect them from the network with a click to contain the incident. They can reverse changes made by threat actors. And if necessary, they can access the system via a remote prompt to check its current state and perform remediation actions. EDR is an invaluable Swiss army knife of response capabilities for defenders.&lt;/p&gt;

&lt;h3 id=&quot;2-edr-is-corporate-spyware&quot;&gt;2. “EDR is corporate spyware”&lt;/h3&gt;

&lt;p&gt;Imagine you’re tasked with protecting your organisation against cyber criminals, ransomware, and other threats - things that if they went unchecked could severely impact operations, cause irrepairable reputational damage, or cost huge sums of money. Ultimately, you and your colleagues’ jobs could all depend on your success. What’s the best tool your could have in your arsenal to detect malicious activity?&lt;/p&gt;

&lt;p&gt;The answer is logs. If you can see exactly which files are written and by whom or what, which processes are run and with which command line arguments, who logged in and where from, and which IP addresses systems are communicating with, you’ll have a huge bank of data to review for signs that something is wrong, and to fully investigate any incidents when they do occur to know exactly what happened.&lt;/p&gt;

&lt;p&gt;At an endpoint level, this is exactly what CrowdStrike and other EDR products do. They watch everything that’s going on, review it for malicious activity, and provide responders with detailed log data that they can pick through to determine the full scope of an incident in order to clean up the mess and put in place improved security measures to prevent the same thing from happening again in future.&lt;/p&gt;

&lt;p&gt;Cyber security professionals sitting in &lt;a href=&quot;/2018/06/01/what-a-security-operations-centre-soc-is-and-how-it-works/&quot; target=&quot;_blank&quot;&gt;security operations centres (SOCs)&lt;/a&gt; have no interest in whether Bob from accounting is spending too much time on Facebook, and frankly, they don’t have the time to check. They are interested solely in investigating threats to the organisation - perhaps if Bob clicked a phishing link in an email or downloaded malware from a site claiming to offer free software.&lt;/p&gt;

&lt;p&gt;In these cases, the logs provided by an EDR tool are invaluable to working out where the threat came from and how much damage it caused. Analysts can use the information within to find any other malicious files on the system and remove them, as well as identifying the exact domain and URL associated with each threat so they can be blocked and other users can’t fall foul of them in future.&lt;/p&gt;

&lt;p&gt;I can understand the confusion when there are so many stories about employers watching workers through webcams and monitoring their mouse movements, but that is not the objective of an EDR product. Cyber security is not HR and it’s not some sort of hi-tech productivity monitoring service - it’s there to protect the company from malicious attacks and to contain any incidents that do occur before they can escalate, and it might well have saved your job at some point without you realising it.&lt;/p&gt;

&lt;h3 id=&quot;3-edr-isnt-worth-the-risk&quot;&gt;3. “EDR isn’t worth the risk”&lt;/h3&gt;

&lt;p&gt;This is by far the most subjective item on this list, and perhaps the hardest to weigh up. If EDR products like CrowdStrike require such low-level, privileged access, and one slip from a vendor can render a system unusable, is it worth the risk to run the security software in the first place?&lt;/p&gt;

&lt;p&gt;I think there are a couple of misconceptions to address for those who would answer “no”. Firstly, despite what many small business leaders might think, your organisation &lt;em&gt;is&lt;/em&gt; a target for cyber criminals. We are past the point where only those holding immense wealth or state secrets are attacked. &lt;em&gt;Your&lt;/em&gt; business needs &lt;em&gt;your&lt;/em&gt; data to operate. Even if the contents of your files are worthless to the threat actors, they know you’re likely to pay up to restore operations if they can manage to deploy ransomware on your network.&lt;/p&gt;

&lt;p&gt;Secondly, this stuff isn’t rare - it happens all the time. For the reasons above, cyber criminals do not need to target specific organisations. Instead, they can scan the internet en masse and fire out “spray-and-pray” campaigns, then work with whatever vulnerabilities are found or whichever users bite. If you’re unaware of any attempts to compromise your organisation then you’re either very lucky or you need to go and pat your security team on the back for the work they’re doing in the background to keep things safe.&lt;/p&gt;

&lt;p&gt;Then there’s the fallout and human impact of an incident when a threat does get through. Having worked in and around incident response for years, I can tell you that a significant incident can be a very traumatic event. It’s not just not being able to work - it’s not being able to pay employees because the payroll server is encrypted, not knowing if you’ll be able to support your family, and in same cases lives lost.&lt;/p&gt;

&lt;p&gt;Cynics will point out that yesterday’s CrowdStrike incident had similar consequences, but I’d encourage you to weigh that against all the times CrowdStrike and other EDR products have &lt;em&gt;prevented&lt;/em&gt; similar scenarios - many instances of which you’re unlikely to be aware of. The bug shouldn’t have made it to production and I’m sure security vendors around the world are now taking a cautious look at their development and deployment processes, but in my opinion we should be careful not to let this one very visible slip-up overshadow all the would-be disasters that were quietly stopped behind the scenes.&lt;/p&gt;</content><author><name>mattcasmith</name></author><summary type="html">When I went to bed on the evening of Friday 19th July, I couldn’t sleep. It was a stuffy summer’s night in London, and the adrenaline was still pumping through my veins after one of the more notable days in recent memory for cyber security. Still, laying awake gave me time to reflect on what I’d seen. Cyber security is a function that normally runs quietly in the background, so when it does make the headlines, it’s often for the wrong reasons. In the past, we’ve seen huge incidents like NotPetya and Wannacry become global news, but yesterday, it was a security product itself that stole the show. What exactly happened? In short: CrowdStrike pushed a bad update to Falcon, a leading endpoint detection and response (EDR) product, that caused a lot of Windows computers to fail to boot. This triggered chaos due to the number of computers around the world running the software. A workaround was quickly discovered, but it needed to be completed manually on each affected system, causing a huge headache for IT administrators. President and CEO George Kurtz took to X to try to reassure the world that no cyber security incident had occurred, and said that CrowdStrike had deployed a fix. He would later share more apologetic posts, including a letter on the firm’s website, but CrowdStrike as a company is not the focus of this post. Cyber security is a cat and mouse game between those who wish to do harm and those who protect our systems, and for this reason security software needs constant updates to detect and prevent the latest attacker techniques. To effectively monitor endpoints, the software must also run in a privileged state, which means that if it fails, it can easily cause more problems than standard user mode applications. Therefore, although CrowdStrike will certainly want to go over its internal testing and review processes, it’s important to remember that a similar thing could happen if any other security vendor made a similar mistake (for example, we saw a similar incident on a smaller scale with Symantec in 2019). Misconceptions and inaccuracies There are a thousand articles out there about the bug itself, how it was fixed, and what this means for CrowdStrike, so I won’t retread that ground here. After the initial shock of what was happening, the next thing that caught my attention was the way CrowdStrike - and by extension EDR as a whole - was being discussed, which showed a complete lack of understanding of the technology and the entire sector. The internet is a terrible place to read about things you’re an expert in, but this trend wasn’t limited to the general population on social media. Even on relatively technical forums like Hacker News there was a strange tone to the conversations, and experienced technology journalists at respected publications dropped huge inaccuracies into their writing. Broadly, these misconceptions fell into three categories:       1. “EDR is anti-virus software”       2. “EDR is corporate spyware”       3. “EDR isn’t worth the risk” I don’t think any of these inaccuracies were spread maliciously. I think they came more from a lack of understanding of what an EDR tool is and does and what cyber security teams are there to do. Therefore, I thought it would be worth writing an article covering the truth behind each of these sentiments. 1. “EDR is anti-virus software” Since home and office computing became mainstream around the 1990s, the vague security threat of “viruses” has loomed. In that era, some of these were actually quite fun. You wouldn’t want them to spread to your PC, but they’d often do quirky things like make all the letters fall to the bottom of your screen or challenge you to a game of cards to save the contents of your hard drive. As malware became more serious, the internet and portable media allowed it to spread more freely, and awareness grew. It became common practice to buy an anti-virus product to identify and stop malicious software, and therefore it was only natural that the name “anti-virus” became a casual synonym for all security software, much like any video game console was sometimes referred to as “a Nintendo”. But the truth is that traditional anti-virus software is quite limited. It relies on hashes and signatures to detect malware, and therefore if the author finds a way to alter their malicious code slightly each time it spreads (polymorphism), it can evade detection. Because it scans for files, standard anti-virus software can also miss attackers’ abuse of legitimate tools, because the files on disk are all benign. The other limitation of traditional anti-virus software is that it usually only provides a small set of automated remediation actions. If it detects malware, it will at most usually terminate any associated processes and quarantine the files. Any further action requires manual access to the computer. EDR tools, including CrowdStrike, do a whole lot more than this. While they do scan files and quarantine them (CrowdStrike even has a component called next-gen anti-virus, or NGAV), to call them “anti-virus” is to sell their capabilities seriously short. EDR products essentially monitor everything that happens on a computer and pick through it with a fine-toothed comb for malicious behaviour - even for threats that do not involve files, like in-memory malware or attackers with direct hands-on-keyboard access. When malicious activity is detected, EDR also gives cyber security professionals many more options to respond to attacks. If it looks like malware could spread between systems, they can disconnect them from the network with a click to contain the incident. They can reverse changes made by threat actors. And if necessary, they can access the system via a remote prompt to check its current state and perform remediation actions. EDR is an invaluable Swiss army knife of response capabilities for defenders. 2. “EDR is corporate spyware” Imagine you’re tasked with protecting your organisation against cyber criminals, ransomware, and other threats - things that if they went unchecked could severely impact operations, cause irrepairable reputational damage, or cost huge sums of money. Ultimately, you and your colleagues’ jobs could all depend on your success. What’s the best tool your could have in your arsenal to detect malicious activity? The answer is logs. If you can see exactly which files are written and by whom or what, which processes are run and with which command line arguments, who logged in and where from, and which IP addresses systems are communicating with, you’ll have a huge bank of data to review for signs that something is wrong, and to fully investigate any incidents when they do occur to know exactly what happened. At an endpoint level, this is exactly what CrowdStrike and other EDR products do. They watch everything that’s going on, review it for malicious activity, and provide responders with detailed log data that they can pick through to determine the full scope of an incident in order to clean up the mess and put in place improved security measures to prevent the same thing from happening again in future. Cyber security professionals sitting in security operations centres (SOCs) have no interest in whether Bob from accounting is spending too much time on Facebook, and frankly, they don’t have the time to check. They are interested solely in investigating threats to the organisation - perhaps if Bob clicked a phishing link in an email or downloaded malware from a site claiming to offer free software. In these cases, the logs provided by an EDR tool are invaluable to working out where the threat came from and how much damage it caused. Analysts can use the information within to find any other malicious files on the system and remove them, as well as identifying the exact domain and URL associated with each threat so they can be blocked and other users can’t fall foul of them in future. I can understand the confusion when there are so many stories about employers watching workers through webcams and monitoring their mouse movements, but that is not the objective of an EDR product. Cyber security is not HR and it’s not some sort of hi-tech productivity monitoring service - it’s there to protect the company from malicious attacks and to contain any incidents that do occur before they can escalate, and it might well have saved your job at some point without you realising it. 3. “EDR isn’t worth the risk” This is by far the most subjective item on this list, and perhaps the hardest to weigh up. If EDR products like CrowdStrike require such low-level, privileged access, and one slip from a vendor can render a system unusable, is it worth the risk to run the security software in the first place? I think there are a couple of misconceptions to address for those who would answer “no”. Firstly, despite what many small business leaders might think, your organisation is a target for cyber criminals. We are past the point where only those holding immense wealth or state secrets are attacked. Your business needs your data to operate. Even if the contents of your files are worthless to the threat actors, they know you’re likely to pay up to restore operations if they can manage to deploy ransomware on your network. Secondly, this stuff isn’t rare - it happens all the time. For the reasons above, cyber criminals do not need to target specific organisations. Instead, they can scan the internet en masse and fire out “spray-and-pray” campaigns, then work with whatever vulnerabilities are found or whichever users bite. If you’re unaware of any attempts to compromise your organisation then you’re either very lucky or you need to go and pat your security team on the back for the work they’re doing in the background to keep things safe. Then there’s the fallout and human impact of an incident when a threat does get through. Having worked in and around incident response for years, I can tell you that a significant incident can be a very traumatic event. It’s not just not being able to work - it’s not being able to pay employees because the payroll server is encrypted, not knowing if you’ll be able to support your family, and in same cases lives lost. Cynics will point out that yesterday’s CrowdStrike incident had similar consequences, but I’d encourage you to weigh that against all the times CrowdStrike and other EDR products have prevented similar scenarios - many instances of which you’re unlikely to be aware of. The bug shouldn’t have made it to production and I’m sure security vendors around the world are now taking a cautious look at their development and deployment processes, but in my opinion we should be careful not to let this one very visible slip-up overshadow all the would-be disasters that were quietly stopped behind the scenes.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mattcasmith.net/wp-content/uploads/2023/01/mcas.png" /><media:content medium="image" url="https://mattcasmith.net/wp-content/uploads/2023/01/mcas.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Centralisation, repeatability, and automation in a modular SOC</title><link href="https://mattcasmith.net/2023/11/26/modular-soc-centralisation-repeatability-automation" rel="alternate" type="text/html" title="Centralisation, repeatability, and automation in a modular SOC" /><published>2023-11-26T00:00:00+00:00</published><updated>2023-11-26T00:00:00+00:00</updated><id>https://mattcasmith.net/2023/11/26/modular-soc-centralisation-repeatability-automation</id><content type="html" xml:base="https://mattcasmith.net/2023/11/26/modular-soc-centralisation-repeatability-automation">&lt;p&gt;The dictionary definition of “modular” leaves a little to be desired: “Employing or involving a module or modules as the basis of design or construction.” What is implied, but that I would make explicit, is that parts of the whole can be swapped out easily while maintaining the functionality of the product.&lt;/p&gt;

&lt;p&gt;Take &lt;a href=&quot;https://frame.work/gb/en&quot; target=&quot;_blank&quot;&gt;Framework&lt;/a&gt;, for example – it bucks the trend for impenetrable, unrepairable, and uncustomisable laptops by introducing a modular design. Want a different processor? Or a keyboard with a numpad? Or even a coloured bezel around your screen? Easy. Everything simply clips in and out, meaning that your device can be adapted over time without the waste of abandoning it for a whole new model.&lt;/p&gt;

&lt;p&gt;The same should apply to your security operations centre (SOC). You’ll always want additional functionality to adapt to advances in technology and changing business requirements, but you should build a foundation that allows for these more tactical changes without wasting time altering the fundamentals, or introducing inefficiency by tacking on extra processes for your team to remember.&lt;/p&gt;

&lt;h3 id=&quot;cra-as-an-approach-to-a-modular-soc&quot;&gt;CRA as an approach to a modular SOC&lt;/h3&gt;

&lt;p&gt;To achieve this, I try to keep three things in mind: centralisation, repeatability, and automation.&lt;/p&gt;

&lt;h4 id=&quot;centralisation&quot;&gt;Centralisation&lt;/h4&gt;

&lt;p&gt;An initial priority should be to develop one queue to rule them all. Rather than having analysts jump between different security tools to look for detections, they should be centralised and integrated into a single pane of glass showing the current state of play across all platforms.&lt;/p&gt;

&lt;p&gt;This doesn’t just apply to detections, but also to communications with your users or customers and also any output from automation tools that must be reviewed. Storing all of this in central tickets or cases not only enables quicker responses to incoming messages and replies, but also ensures the information is easily available to analysts working on similar detections and queries in future.&lt;/p&gt;

&lt;p&gt;This setup also means analysts themselves can be modular - not in such a way that individuals are viewed as interchangeable in their roles, but in a way that any ticket that enters (or returns to) the queue can be claimed by any available analyst, empowering them to work more efficiently and exposing them to a greater variety of scenarios. For example, by picking up a reply to an existing ticket, a more junior analyst has the opportunity to learn about the analysis performed by the original owner.&lt;/p&gt;

&lt;h4 id=&quot;repeatability&quot;&gt;Repeatability&lt;/h4&gt;

&lt;p&gt;Supporting the technology should be processes, templates, and so on that ensure that work is completed consistently, and providing less experienced analysts with documentation to refer to for unfamiliar case types. Templates are huge timesavers for any content that analysts must compose multiple times per day, and pre-cooked log queries and scripts can help to cut the time spent on common investigations.&lt;/p&gt;

&lt;p&gt;Centralisation helps here as it reduces the number of processes required - for example, if closing a case in the security orchestration, automation, and response (SOAR) tool automatically closes detections via an API, analysts only need to know how to do that, and not how to resolve detections in tools A, B, and C.&lt;/p&gt;

&lt;p&gt;By this point, onboarding a new tool should be as simple as performing an API integration and producing response processes for any new detection types. Once detections and tickets are centralised and processes are recorded, the workflow for each can become so standardised that it may even be possible (in theory, at least) to map out the entire analyst-facing process into a single flowchart.&lt;/p&gt;

&lt;h4 id=&quot;automation&quot;&gt;Automation&lt;/h4&gt;

&lt;p&gt;A great way to save analysts’ time is to automate basic, often-repeated response processes, but even before that the easiest and most impactful automation workflows are often those associated with your SOC’s housekeeping - agent health checks, weekly metrics, sweeps for unresolved detections, and so on.&lt;/p&gt;

&lt;p&gt;Further down the line, of course, your automation suite will extend into more complex and specialised processes - automatic triage and remediation of commonly seen types of malware, for example. At this point the testing process becomes critical, because a wrong step during eradication can be disastrous.&lt;/p&gt;

&lt;p&gt;You might be thinking, “How can I ever fully trust the automation logic to make the right call all the time?” But to this I’d argue that even partial automation is more efficient than none at all. Sending output to a ticket for an analyst to review, or running a triage script as a first step to provide the basis for an analyst to perform remediation on the discovered artefacts, still saves time and makes people’s jobs easier.&lt;/p&gt;

&lt;h3 id=&quot;adapting-and-expanding&quot;&gt;Adapting and expanding&lt;/h3&gt;

&lt;p&gt;Once the above has been implemented, even to a basic level, adapting SOC operations to new challenges becomes far easier than it would be in a more manual, less consolidated environment.&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Procured a new security tool?&lt;/strong&gt; Perform the relevant API integration with your SOAR so its detections appear in the queue, and build out any basic automation workflows and processes required&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Analyst working a big case off sick?&lt;/strong&gt; No problem - when the customer next replies to the ticket it’ll pop right into the queue, with all the relevant infromation available to the next available analyst&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Swamped by a new, widespread type of malware?&lt;/strong&gt; Build out triage and/or remediation scripts and set up a workflow whereby your SOAR runs them automatically for relevant detections&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Onboarding a new part of the organisation?&lt;/strong&gt; Once its tools are integrated via their APIs and the relevant escalation contacts are centrally stored, it’ll slot right in alongside your existing customers&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Both IT and cyber security are rapidly changing fields, and therefore it makes sense to build a SOC that is agile and adaptable to those changes. By structuring your SOC with modularity in mind, you’re putting yourself and your team in the best position to respond swiftly when change inevitably occurs.&lt;/p&gt;</content><author><name>mattcasmith</name></author><summary type="html">The dictionary definition of “modular” leaves a little to be desired: “Employing or involving a module or modules as the basis of design or construction.” What is implied, but that I would make explicit, is that parts of the whole can be swapped out easily while maintaining the functionality of the product. Take Framework, for example – it bucks the trend for impenetrable, unrepairable, and uncustomisable laptops by introducing a modular design. Want a different processor? Or a keyboard with a numpad? Or even a coloured bezel around your screen? Easy. Everything simply clips in and out, meaning that your device can be adapted over time without the waste of abandoning it for a whole new model. The same should apply to your security operations centre (SOC). You’ll always want additional functionality to adapt to advances in technology and changing business requirements, but you should build a foundation that allows for these more tactical changes without wasting time altering the fundamentals, or introducing inefficiency by tacking on extra processes for your team to remember. CRA as an approach to a modular SOC To achieve this, I try to keep three things in mind: centralisation, repeatability, and automation. Centralisation An initial priority should be to develop one queue to rule them all. Rather than having analysts jump between different security tools to look for detections, they should be centralised and integrated into a single pane of glass showing the current state of play across all platforms. This doesn’t just apply to detections, but also to communications with your users or customers and also any output from automation tools that must be reviewed. Storing all of this in central tickets or cases not only enables quicker responses to incoming messages and replies, but also ensures the information is easily available to analysts working on similar detections and queries in future. This setup also means analysts themselves can be modular - not in such a way that individuals are viewed as interchangeable in their roles, but in a way that any ticket that enters (or returns to) the queue can be claimed by any available analyst, empowering them to work more efficiently and exposing them to a greater variety of scenarios. For example, by picking up a reply to an existing ticket, a more junior analyst has the opportunity to learn about the analysis performed by the original owner. Repeatability Supporting the technology should be processes, templates, and so on that ensure that work is completed consistently, and providing less experienced analysts with documentation to refer to for unfamiliar case types. Templates are huge timesavers for any content that analysts must compose multiple times per day, and pre-cooked log queries and scripts can help to cut the time spent on common investigations. Centralisation helps here as it reduces the number of processes required - for example, if closing a case in the security orchestration, automation, and response (SOAR) tool automatically closes detections via an API, analysts only need to know how to do that, and not how to resolve detections in tools A, B, and C. By this point, onboarding a new tool should be as simple as performing an API integration and producing response processes for any new detection types. Once detections and tickets are centralised and processes are recorded, the workflow for each can become so standardised that it may even be possible (in theory, at least) to map out the entire analyst-facing process into a single flowchart. Automation A great way to save analysts’ time is to automate basic, often-repeated response processes, but even before that the easiest and most impactful automation workflows are often those associated with your SOC’s housekeeping - agent health checks, weekly metrics, sweeps for unresolved detections, and so on. Further down the line, of course, your automation suite will extend into more complex and specialised processes - automatic triage and remediation of commonly seen types of malware, for example. At this point the testing process becomes critical, because a wrong step during eradication can be disastrous. You might be thinking, “How can I ever fully trust the automation logic to make the right call all the time?” But to this I’d argue that even partial automation is more efficient than none at all. Sending output to a ticket for an analyst to review, or running a triage script as a first step to provide the basis for an analyst to perform remediation on the discovered artefacts, still saves time and makes people’s jobs easier. Adapting and expanding Once the above has been implemented, even to a basic level, adapting SOC operations to new challenges becomes far easier than it would be in a more manual, less consolidated environment. Procured a new security tool? Perform the relevant API integration with your SOAR so its detections appear in the queue, and build out any basic automation workflows and processes required Analyst working a big case off sick? No problem - when the customer next replies to the ticket it’ll pop right into the queue, with all the relevant infromation available to the next available analyst Swamped by a new, widespread type of malware? Build out triage and/or remediation scripts and set up a workflow whereby your SOAR runs them automatically for relevant detections Onboarding a new part of the organisation? Once its tools are integrated via their APIs and the relevant escalation contacts are centrally stored, it’ll slot right in alongside your existing customers Both IT and cyber security are rapidly changing fields, and therefore it makes sense to build a SOC that is agile and adaptable to those changes. By structuring your SOC with modularity in mind, you’re putting yourself and your team in the best position to respond swiftly when change inevitably occurs.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mattcasmith.net/wp-content/uploads/2023/01/mcas.png" /><media:content medium="image" url="https://mattcasmith.net/wp-content/uploads/2023/01/mcas.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Cyber security sometimes means learning things backwards</title><link href="https://mattcasmith.net/2023/03/26/cyber-security-learning-things-backwards" rel="alternate" type="text/html" title="Cyber security sometimes means learning things backwards" /><published>2023-03-26T00:00:00+00:00</published><updated>2023-03-26T00:00:00+00:00</updated><id>https://mattcasmith.net/2023/03/26/cyber-security-learning-things-backwards</id><content type="html" xml:base="https://mattcasmith.net/2023/03/26/cyber-security-learning-things-backwards">&lt;p&gt;Stick around cyber security Twitter or LinkedIn for long enough and you’ll likely see somebody raise a question about how to get into the industry. You’ll also likely see a reply that describes a kind of rite of passage from sysadmin, to SOC analyst, to just about any other security role.&lt;/p&gt;

&lt;p&gt;The logic behind this - that you need to know how systems work in order to defend them - is sound, but I feel like the entry requirements are often exaggerated as a form of gatekeeping, and that years in a full-on sysadmin role aren’t necessarily required to investigate cyber attacks effectively. There are other paths.&lt;/p&gt;

&lt;h3 id=&quot;learning-the-ropes&quot;&gt;Learning the ropes&lt;/h3&gt;

&lt;p&gt;I’m a good example here. From childhood I was &lt;em&gt;vaguely&lt;/em&gt; technical in a hobbyist sense. That was enough for me to peer through the glass into the tech world and get a rough feeling for how computers work, but I certainly wasn’t administrating corporate Active Directory domains during my years as a journalist.&lt;/p&gt;

&lt;p&gt;Some education was required. At the &lt;a href=&quot;/2017/03/27/finishing-line-ive-passed-my-gcih-exam/&quot;&gt;SANS Cyber Retraining Academy&lt;/a&gt; I took courses that equipped me with fundamentals covering operating systems, networking, and components of business environments that I hadn’t had any exposure to in my days tinkering with my home computer. It took the scattered knowledge I’d accumulated, filled in the blanks, and gave it better structure and context.&lt;/p&gt;

&lt;p&gt;One of the best decisions I made around that time was to teach myself Python and the Linux command line. Do I use these specific tools every day now? No. But defensive PowerShell scripts, malicious JavaScript files, and commands run by threat actors all use very similar syntax, to a level that I could decode and roughly understand them even before I properly taught myself the languages involved.&lt;/p&gt;

&lt;h3 id=&quot;beyond-the-basics&quot;&gt;Beyond the basics&lt;/h3&gt;

&lt;p&gt;Not everything can be learnt in the classroom, and I’ve written peviously about how &lt;a href=&quot;/2018/01/20/what-cyber-security-courses-dont-prepare-you-for/&quot;&gt;applying foundational knowledge outside of a lab environment takes some practice&lt;/a&gt;, but with the basics under your belt, you can sometimes learn just as much about the way technology works by examining the way threat actors try to abuse it as you can by building and configuring the systems yourself.&lt;/p&gt;

&lt;p&gt;The key as far as I’m concerned isn’t years on the helpdesk (although that won’t hurt), it’s staying curious and doing more digging than your day job requires. Let your investigations and administrative duties point you in a useful direction (who better to show you what you need to learn to stop attackers than the attackers themselves?) but stick with the material and go deeper than you need to.&lt;/p&gt;

&lt;p&gt;Eventually you’ll develop a computing literacy that extends beyond your immediate knowledge. As an example, I recently helped an administrator to troubleshoot an Active Directory &lt;a href=&quot;https://learn.microsoft.com/en-us/previous-versions/windows/desktop/policy/group-policy-objects&quot; target=&quot;_blank&quot;&gt;group policy object (GPO)&lt;/a&gt;. As a non-sysadmin, I’d never configured one myself, but my training taught me &lt;em&gt;what&lt;/em&gt; a GPO is, I’d seen enough to know what they look like, and some quick research was enough to work out the specifics.&lt;/p&gt;

&lt;p&gt;Cyber security - particularly on the defensive side, where your work is rarely planned ahead - can throw practically anything at you, and it’s impossible to learn everything to the depth that an administrator looking after a certain type of system would. The best approach is to know your basics and stay curious to continuously grow and build on top of that. This way you’ll be clued up enough to understand the documentation (and, let’s face it, your Google results) when something new lands on your desk.&lt;/p&gt;</content><author><name>mattcasmith</name></author><summary type="html">Stick around cyber security Twitter or LinkedIn for long enough and you’ll likely see somebody raise a question about how to get into the industry. You’ll also likely see a reply that describes a kind of rite of passage from sysadmin, to SOC analyst, to just about any other security role. The logic behind this - that you need to know how systems work in order to defend them - is sound, but I feel like the entry requirements are often exaggerated as a form of gatekeeping, and that years in a full-on sysadmin role aren’t necessarily required to investigate cyber attacks effectively. There are other paths. Learning the ropes I’m a good example here. From childhood I was vaguely technical in a hobbyist sense. That was enough for me to peer through the glass into the tech world and get a rough feeling for how computers work, but I certainly wasn’t administrating corporate Active Directory domains during my years as a journalist. Some education was required. At the SANS Cyber Retraining Academy I took courses that equipped me with fundamentals covering operating systems, networking, and components of business environments that I hadn’t had any exposure to in my days tinkering with my home computer. It took the scattered knowledge I’d accumulated, filled in the blanks, and gave it better structure and context. One of the best decisions I made around that time was to teach myself Python and the Linux command line. Do I use these specific tools every day now? No. But defensive PowerShell scripts, malicious JavaScript files, and commands run by threat actors all use very similar syntax, to a level that I could decode and roughly understand them even before I properly taught myself the languages involved. Beyond the basics Not everything can be learnt in the classroom, and I’ve written peviously about how applying foundational knowledge outside of a lab environment takes some practice, but with the basics under your belt, you can sometimes learn just as much about the way technology works by examining the way threat actors try to abuse it as you can by building and configuring the systems yourself. The key as far as I’m concerned isn’t years on the helpdesk (although that won’t hurt), it’s staying curious and doing more digging than your day job requires. Let your investigations and administrative duties point you in a useful direction (who better to show you what you need to learn to stop attackers than the attackers themselves?) but stick with the material and go deeper than you need to. Eventually you’ll develop a computing literacy that extends beyond your immediate knowledge. As an example, I recently helped an administrator to troubleshoot an Active Directory group policy object (GPO). As a non-sysadmin, I’d never configured one myself, but my training taught me what a GPO is, I’d seen enough to know what they look like, and some quick research was enough to work out the specifics. Cyber security - particularly on the defensive side, where your work is rarely planned ahead - can throw practically anything at you, and it’s impossible to learn everything to the depth that an administrator looking after a certain type of system would. The best approach is to know your basics and stay curious to continuously grow and build on top of that. This way you’ll be clued up enough to understand the documentation (and, let’s face it, your Google results) when something new lands on your desk.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mattcasmith.net/wp-content/uploads/2023/01/mcas.png" /><media:content medium="image" url="https://mattcasmith.net/wp-content/uploads/2023/01/mcas.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Using winget to automate software deployment to a new laptop</title><link href="https://mattcasmith.net/2023/02/17/winget-automate-software-deployment-new-laptop" rel="alternate" type="text/html" title="Using winget to automate software deployment to a new laptop" /><published>2023-02-17T00:00:00+00:00</published><updated>2023-02-17T00:00:00+00:00</updated><id>https://mattcasmith.net/2023/02/17/winget-automate-software-deployment-new-laptop</id><content type="html" xml:base="https://mattcasmith.net/2023/02/17/winget-automate-software-deployment-new-laptop">&lt;p&gt;I got my first new laptop in six years this week! The new hardware is definitely exciting, but reviews aren’t really my thing, so while I’ll inevitably &lt;a href=&quot;https://twitter.com/mattcasmith&quot; target=&quot;_blank&quot;&gt;tweet&lt;/a&gt; about how the Microsoft Surface Laptop fares, that’s not the purpose of this post. This is more about the mundane job of porting all my usual applications over to a new PC, and a handy new Windows feature that helped to make things a little easier - &lt;code&gt;winget&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Before we get started, I want to acknowledge both that I’m &lt;a href=&quot;https://www.bleepingcomputer.com/news/security/winget-how-to-use-windows-10s-new-native-package-manager/&quot; target=&quot;_blank&quot;&gt;late to the party&lt;/a&gt; on &lt;code&gt;winget&lt;/code&gt; and that there have been &lt;a href=&quot;https://chocolatey.org/&quot; target=&quot;_blank&quot;&gt;package managers&lt;/a&gt; around on Windows for years - but it’s certainly a cool and useful addition to the core operating system, which bucks the trend when you consider how frequently Microsoft &lt;em&gt;removes&lt;/em&gt; more advanced features and options these days (or at least applies confusing double UIs).&lt;/p&gt;

&lt;p&gt;I’ll also concede that taking the time to write a PowerShell script to automate software deployment is probably overkill for most users - myself included. The time spent on development and testing is &lt;em&gt;probably&lt;/em&gt; greater than the amount of time it would take to download and install each package individually, but at the end of the day I now have a pre-baked script I can use to drop my software on any future systems.&lt;/p&gt;

&lt;h3 id=&quot;initial-research&quot;&gt;Initial research&lt;/h3&gt;

&lt;p&gt;Before I even got as far as creating a &lt;code&gt;PS1&lt;/code&gt; file, my first step was to do some research to identify the software packages I wanted to install. To achieve this, I used the following &lt;code&gt;winget&lt;/code&gt; command.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;n&quot;&gt;winget&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;search&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;APP_NAME&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;This ran a search across all the configured repositories (by default, basically the Microsoft Store and &lt;code&gt;winget&lt;/code&gt;’s own one). The most important details were in the &lt;code&gt;Id&lt;/code&gt; column, which generally follows the format &lt;code&gt;Developer.Product&lt;/code&gt;. There were a lot of third-party apps that had the names of popular software in their own names, so this helped to sort the wheat from the chaff and identify the real packages.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2023/02/winget_0.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;During this planning stage, I considered the software I wanted to deploy to my new system and ran the &lt;code&gt;winget search&lt;/code&gt; command to find the corresponding packages until I had a complete list.&lt;/p&gt;

&lt;h3 id=&quot;the-powershell-script&quot;&gt;The PowerShell script&lt;/h3&gt;

&lt;p&gt;Now I could start writing a PowerShell script to iterate through my list of applications and try to install each of them. Firstly, I created an array to store the list (note that only the &lt;code&gt;Id&lt;/code&gt; values here match the repository data - the names are my own more readable ones, for display and logging only).&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;err&quot;&gt;$&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;install_apps&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; 
    &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Firefox&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;Mozilla.Firefox&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Steam&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;Valve.Steam&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Discord&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;Discord.Discord&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Netflix&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;9WZDNCRFJ3TJ&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Spotify&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;Spotify.Spotify&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;Next, I used a &lt;code&gt;foreach&lt;/code&gt; loop to work through each application in &lt;code&gt;$install_apps&lt;/code&gt; and attempt to run &lt;code&gt;winget&lt;/code&gt; to install the corresponding package according to its &lt;code&gt;Id&lt;/code&gt;, using the &lt;code&gt;--silent&lt;/code&gt; flag to keep output to a minimum and require as little user interaction as possible to complete the deployment.&lt;/p&gt;

&lt;p&gt;There are a number of reasons why installation of any given application might fail - a missing package or insufficient permissions to name just a couple. To prevent missing software from flying under the radar, I used an &lt;code&gt;if&lt;/code&gt;/&lt;code&gt;else&lt;/code&gt; statement based on the &lt;code&gt;winget&lt;/code&gt; exit code (&lt;code&gt;$LASTEXITCODE&lt;/code&gt;) to detect any issues.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;n&quot;&gt;foreach&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;$&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;app&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;err&quot;&gt;$&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;install_apps&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;Write&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Host&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;Installing&quot;&lt;/span&gt; &lt;span class=&quot;err&quot;&gt;$&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;app&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;...&quot;&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;winget&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;install&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;--&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;silent&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;--&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;id&lt;/span&gt; &lt;span class=&quot;err&quot;&gt;$&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;app&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;$&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;LASTEXITCODE&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;eq&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;Write&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Host&lt;/span&gt; &lt;span class=&quot;err&quot;&gt;$&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;app&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;installation complete.&quot;&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;Write&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Host&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;&quot;&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;else&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;Write&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Host&lt;/span&gt; &lt;span class=&quot;err&quot;&gt;$&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;app&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;installation failed.&quot;&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;Write&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Host&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;&quot;&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;And that’s pretty much all I needed to get going! I also added a section to install Windows Subsystem for Linux (with Ubuntu) and a few extra lines to log failures to a file (in case I missed them in the terminal), but the code above would be perfectly adequate to try to install all the packages in the given list.&lt;/p&gt;

&lt;h3 id=&quot;the-results&quot;&gt;The results&lt;/h3&gt;

&lt;p&gt;So how did things go with the new laptop? Did my &lt;code&gt;winget&lt;/code&gt; PowerShell script solve all my problems and allow me to kick back and relax as my applications were automatically deployed to my new system?&lt;/p&gt;

&lt;p&gt;Well, to tell the complete truth, not quite… But it was very close.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2023/02/winget_1.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;As you can see from the screenshot above, &lt;code&gt;winget&lt;/code&gt; undoubtedly made the job much quicker and easier, but it wasn’t &lt;em&gt;entirely&lt;/em&gt; hands-off. Spotify was the only app that failed to install for some reason (perhaps because it’s one of those apps that come half-installed on a fresh Windows system?), but there were also a couple of others that required me to hit the &lt;code&gt;Y&lt;/code&gt; key to accept licence agreements and terms of use.&lt;/p&gt;

&lt;p style=&quot;float: left; width: 90%; padding: 5px; padding-left: 15px; padding-right: 5%; margin: 30px 0 10px 0; font-style: italic; border-left: 5px solid #FFFF00; background: #FFFFB9; color: #333;&quot;&gt;I eventually found a solution to this part of the issue - unfortunately just after I ran the script. You can accept licence agreements automatically by using the &lt;code&gt;--accept-source-agreements&lt;/code&gt; and &lt;code&gt;--accept-package-agreements&lt;/code&gt; parameters when running &lt;code&gt;winget&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Still, it was a fun experiment, saved a lot of time downloading installers from developer websites and running them manually via the GUI, and leaves me with a script that will be easy to repurpose to (almost) automate the setup of any new computers I adopt in future. While &lt;code&gt;winget&lt;/code&gt; isn’t the perfect zero-touch solution, I’d definitely recommend giving it a go - particularly if you ever find yourself in a situation where you need to set up multiple systems in a short time and a cheat code would help to preserve your sanity.&lt;/p&gt;</content><author><name>mattcasmith</name></author><summary type="html">I got my first new laptop in six years this week! The new hardware is definitely exciting, but reviews aren’t really my thing, so while I’ll inevitably tweet about how the Microsoft Surface Laptop fares, that’s not the purpose of this post. This is more about the mundane job of porting all my usual applications over to a new PC, and a handy new Windows feature that helped to make things a little easier - winget. Before we get started, I want to acknowledge both that I’m late to the party on winget and that there have been package managers around on Windows for years - but it’s certainly a cool and useful addition to the core operating system, which bucks the trend when you consider how frequently Microsoft removes more advanced features and options these days (or at least applies confusing double UIs). I’ll also concede that taking the time to write a PowerShell script to automate software deployment is probably overkill for most users - myself included. The time spent on development and testing is probably greater than the amount of time it would take to download and install each package individually, but at the end of the day I now have a pre-baked script I can use to drop my software on any future systems. Initial research Before I even got as far as creating a PS1 file, my first step was to do some research to identify the software packages I wanted to install. To achieve this, I used the following winget command. winget search &amp;lt;APP_NAME&amp;gt; This ran a search across all the configured repositories (by default, basically the Microsoft Store and winget’s own one). The most important details were in the Id column, which generally follows the format Developer.Product. There were a lot of third-party apps that had the names of popular software in their own names, so this helped to sort the wheat from the chaff and identify the real packages. During this planning stage, I considered the software I wanted to deploy to my new system and ran the winget search command to find the corresponding packages until I had a complete list. The PowerShell script Now I could start writing a PowerShell script to iterate through my list of applications and try to install each of them. Firstly, I created an array to store the list (note that only the Id values here match the repository data - the names are my own more readable ones, for display and logging only). $install_apps = (&quot;Firefox&quot;, &quot;Mozilla.Firefox&quot;), (&quot;Steam&quot;, &quot;Valve.Steam&quot;), (&quot;Discord&quot;, &quot;Discord.Discord&quot;), (&quot;Netflix&quot;, &quot;9WZDNCRFJ3TJ&quot;), (&quot;Spotify&quot;, &quot;Spotify.Spotify&quot;) Next, I used a foreach loop to work through each application in $install_apps and attempt to run winget to install the corresponding package according to its Id, using the --silent flag to keep output to a minimum and require as little user interaction as possible to complete the deployment. There are a number of reasons why installation of any given application might fail - a missing package or insufficient permissions to name just a couple. To prevent missing software from flying under the radar, I used an if/else statement based on the winget exit code ($LASTEXITCODE) to detect any issues. foreach ($app in $install_apps) { Write-Host &quot;Installing&quot; $app[0] &quot;...&quot; winget install --silent --id $app[1] if ($LASTEXITCODE -eq 0) { Write-Host $app[0] &quot;installation complete.&quot; Write-Host &quot;&quot; } else { Write-Host $app[0] &quot;installation failed.&quot; Write-Host &quot;&quot; } } And that’s pretty much all I needed to get going! I also added a section to install Windows Subsystem for Linux (with Ubuntu) and a few extra lines to log failures to a file (in case I missed them in the terminal), but the code above would be perfectly adequate to try to install all the packages in the given list. The results So how did things go with the new laptop? Did my winget PowerShell script solve all my problems and allow me to kick back and relax as my applications were automatically deployed to my new system? Well, to tell the complete truth, not quite… But it was very close. As you can see from the screenshot above, winget undoubtedly made the job much quicker and easier, but it wasn’t entirely hands-off. Spotify was the only app that failed to install for some reason (perhaps because it’s one of those apps that come half-installed on a fresh Windows system?), but there were also a couple of others that required me to hit the Y key to accept licence agreements and terms of use. I eventually found a solution to this part of the issue - unfortunately just after I ran the script. You can accept licence agreements automatically by using the --accept-source-agreements and --accept-package-agreements parameters when running winget. Still, it was a fun experiment, saved a lot of time downloading installers from developer websites and running them manually via the GUI, and leaves me with a script that will be easy to repurpose to (almost) automate the setup of any new computers I adopt in future. While winget isn’t the perfect zero-touch solution, I’d definitely recommend giving it a go - particularly if you ever find yourself in a situation where you need to set up multiple systems in a short time and a cheat code would help to preserve your sanity.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mattcasmith.net/wp-content/uploads/2023/02/winget_1.png" /><media:content medium="image" url="https://mattcasmith.net/wp-content/uploads/2023/02/winget_1.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">2023: Thoughts on new challenges and sharing experiences</title><link href="https://mattcasmith.net/2023/01/13/2023-new-challenges-sharing-experiences" rel="alternate" type="text/html" title="2023&amp;#58; Thoughts on new challenges and sharing experiences" /><published>2023-01-13T00:00:00+00:00</published><updated>2023-01-13T00:00:00+00:00</updated><id>https://mattcasmith.net/2023/01/13/2023-new-challenges-sharing-experiences</id><content type="html" xml:base="https://mattcasmith.net/2023/01/13/2023-new-challenges-sharing-experiences">&lt;p&gt;A belated happy new year! If you’re reading this, I hope you have a terrific 2023.&lt;/p&gt;

&lt;p&gt;I recently went back and read some old posts that have long since been deleted from this blog - writings from my journalism days that reminded me of my perspectives on certain news events and emerging technologies from the mid-2010s and sparked some nostalgia for my early twenties along the way.&lt;/p&gt;

&lt;p&gt;By the end of 2017, I’d landed a job in cyber security and the tone of my blog changed. Gone were the slightly rambling posts that gave a sense of personality, and what came was primarily content consisting of technical notes about things I’d been working on in &lt;a href=&quot;/category/programming&quot;&gt;Python&lt;/a&gt; or a particular &lt;a href=&quot;/category/cyber-security&quot;&gt;forensic artefact&lt;/a&gt; (don’t get me wrong - I’m proud of many of these articles, and they remain a useful way of organising my thoughts).&lt;/p&gt;

&lt;h3 id=&quot;reasons-for-retreating&quot;&gt;Reasons for retreating&lt;/h3&gt;

&lt;p&gt;My change of industry played a role in this. Despite having reported on cyber security for four years beforehand, my hesitance over ranting about a sector I was quite new to in the same way I’d written about consumer technology was justified. Another factor was that my job just wasn’t as public - an interesting conversation or briefing as a journalist might make a good blog post if it didn’t fit my employer’s site, but accounts of incidents I work and security discussions with clients must stay behind closed doors.&lt;/p&gt;

&lt;p&gt;But there was also a force at play that was more toxic than humbleness or confidentiality: anxiety. I’d seen flashes of it before as I battled imposter syndrome on entering cyber security, but the real thing crept up on me during the COVID-19 pandemic, and I spent a good deal of 2022 pushing myself in various ways to get it under control. It still has its occasional moments, but I’m in a much better place heading into 2023.&lt;/p&gt;

&lt;p&gt;While keeping background stress as low as possible helped with this, I was also anxious in less of a clinical sense about writing. The less you put yourself out there, the less likely it is that you’ll be called out on something, or somebody will tell you that you’re wrong. Inertia is like laying in a safety net.&lt;/p&gt;

&lt;h3 id=&quot;incremental-improvement&quot;&gt;Incremental improvement&lt;/h3&gt;

&lt;p&gt;But if you’re sitting still on the net, paralysed out of fear of action and its consequences, you’ll never walk the tightrope. Sure, it’s far less likely that you’ll fail, but you’ll also never do anything remarkable. For all we know, the greatest ideas in history might not be the ones in modern day textbooks - they might never have made it that far, living and dying inside their creators’ heads because they were never shared.&lt;/p&gt;

&lt;p&gt;I doubt anything I write on these pages will change the world, and most of my ideas are iterations on or combinations of existing ones, rather than fully original thoughts. But even if I have nothing revolutonary to say, the little things that inspire me to put digital pen to paper could still be enough to make our small corner of the world a better place, even if it’s just a tip that saves someone else five minutes.&lt;/p&gt;

&lt;p&gt;Some changes in my life have put me what will hopefully be a far better place to trial ideas and implement change in 2023, and I intend to write some more personal, thoughtful posts to share my learnings from these experiments. I still might not be able to go into as much detail as I did a decade ago, but I’m feeling energetic, inspired, and more motivated to share my experiences than I have been in years.&lt;/p&gt;

&lt;p&gt;If you’re interested, check &lt;a href=&quot;https://mattcasmith.net&quot;&gt;my blog&lt;/a&gt; regularly for new posts, and &lt;a href=&quot;https://twitter.com/mattcasmith&quot; target=&quot;_blank&quot;&gt;follow me on Twitter&lt;/a&gt; for day-to-day updates. Feel free to &lt;a href=&quot;mailto:mattcasmith@protonmail.com&quot;&gt;email me&lt;/a&gt;, too - it’s an exciting, optimistic time and I’d be happy to connect.&lt;/p&gt;</content><author><name>mattcasmith</name></author><summary type="html">A belated happy new year! If you’re reading this, I hope you have a terrific 2023. I recently went back and read some old posts that have long since been deleted from this blog - writings from my journalism days that reminded me of my perspectives on certain news events and emerging technologies from the mid-2010s and sparked some nostalgia for my early twenties along the way. By the end of 2017, I’d landed a job in cyber security and the tone of my blog changed. Gone were the slightly rambling posts that gave a sense of personality, and what came was primarily content consisting of technical notes about things I’d been working on in Python or a particular forensic artefact (don’t get me wrong - I’m proud of many of these articles, and they remain a useful way of organising my thoughts). Reasons for retreating My change of industry played a role in this. Despite having reported on cyber security for four years beforehand, my hesitance over ranting about a sector I was quite new to in the same way I’d written about consumer technology was justified. Another factor was that my job just wasn’t as public - an interesting conversation or briefing as a journalist might make a good blog post if it didn’t fit my employer’s site, but accounts of incidents I work and security discussions with clients must stay behind closed doors. But there was also a force at play that was more toxic than humbleness or confidentiality: anxiety. I’d seen flashes of it before as I battled imposter syndrome on entering cyber security, but the real thing crept up on me during the COVID-19 pandemic, and I spent a good deal of 2022 pushing myself in various ways to get it under control. It still has its occasional moments, but I’m in a much better place heading into 2023. While keeping background stress as low as possible helped with this, I was also anxious in less of a clinical sense about writing. The less you put yourself out there, the less likely it is that you’ll be called out on something, or somebody will tell you that you’re wrong. Inertia is like laying in a safety net. Incremental improvement But if you’re sitting still on the net, paralysed out of fear of action and its consequences, you’ll never walk the tightrope. Sure, it’s far less likely that you’ll fail, but you’ll also never do anything remarkable. For all we know, the greatest ideas in history might not be the ones in modern day textbooks - they might never have made it that far, living and dying inside their creators’ heads because they were never shared. I doubt anything I write on these pages will change the world, and most of my ideas are iterations on or combinations of existing ones, rather than fully original thoughts. But even if I have nothing revolutonary to say, the little things that inspire me to put digital pen to paper could still be enough to make our small corner of the world a better place, even if it’s just a tip that saves someone else five minutes. Some changes in my life have put me what will hopefully be a far better place to trial ideas and implement change in 2023, and I intend to write some more personal, thoughtful posts to share my learnings from these experiments. I still might not be able to go into as much detail as I did a decade ago, but I’m feeling energetic, inspired, and more motivated to share my experiences than I have been in years. If you’re interested, check my blog regularly for new posts, and follow me on Twitter for day-to-day updates. Feel free to email me, too - it’s an exciting, optimistic time and I’d be happy to connect.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mattcasmith.net/wp-content/uploads/2023/01/mcas.png" /><media:content medium="image" url="https://mattcasmith.net/wp-content/uploads/2023/01/mcas.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Investigating Explorer’s temporary ZIP folders and retrieving files</title><link href="https://mattcasmith.net/2022/12/15/investigating-explorer-temporary-zip-folders" rel="alternate" type="text/html" title="Investigating Explorer's temporary ZIP folders and retrieving files" /><published>2022-12-14T00:00:00+00:00</published><updated>2022-12-14T00:00:00+00:00</updated><id>https://mattcasmith.net/2022/12/15/investigating-explorer-temporary-zip-folders</id><content type="html" xml:base="https://mattcasmith.net/2022/12/15/investigating-explorer-temporary-zip-folders">&lt;p&gt;If I was to describe how often malware is downloaded within ZIP archives, “common” would be a huge understatement. A key artefact in these investigations is the temporary directory Windows creates when a user opens an archive in Explorer, but I recently realised I’d never actually run a proper test to see when the folder is created, when it is not, and when it is deleted. So to clear that up, here’s a quick blog post.&lt;/p&gt;

&lt;h3 id=&quot;zip-archives-on-easy-mode&quot;&gt;ZIP archives on easy mode&lt;/h3&gt;

&lt;p&gt;There was a time when a ZIP archive was an indecipherable box that Windows couldn’t see inside. To access the files within, you’d have to download third-party software, like a trial version of WinZip that would incessantly prompt you to buy a licence whenever you downloaded something from the internet.&lt;/p&gt;

&lt;p&gt;That all came to an end with - I &lt;em&gt;think&lt;/em&gt;, although I may be slightly mistaken - Windows XP, when Microsoft added native capabilities within Windows Explorer to zip and unzip files. This is now a near-seamless experience, with the user able to open a ZIP archive as though it were a normal folder, as seen below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/12/explorer-temporary-zip-folders_1.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;But what’s going on behind the scenes when this happens, and what artefacts does it leave behind for forensicators and responders? With so much malware downloaded from the internet and attached to emails inside ZIP archives, knowing where to look can be a powerful tool in tracing user actions.&lt;/p&gt;

&lt;h3 id=&quot;the-amazing-disappearing-folder&quot;&gt;The amazing disappearing folder&lt;/h3&gt;

&lt;p&gt;This focus of this post is the temporary directory that Explorer creates, quietly unzipping parts of the archive on demand in the background so that it can present the user with that seamless interface on the front end. It’s located within the user’s &lt;code&gt;AppData&lt;/code&gt; &lt;code&gt;Temp&lt;/code&gt; folder, under a path like this:&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;n&quot;&gt;C&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;\&lt;span class=&quot;n&quot;&gt;Users&lt;/span&gt;\&lt;span class=&quot;n&quot;&gt;user&lt;/span&gt;\&lt;span class=&quot;n&quot;&gt;AppData&lt;/span&gt;\&lt;span class=&quot;n&quot;&gt;Local&lt;/span&gt;\&lt;span class=&quot;n&quot;&gt;Temp&lt;/span&gt;\&lt;span class=&quot;n&quot;&gt;Temp1_zipname&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;zip&lt;/span&gt;\&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;But when exactly is the folder created? I previously assumed, wrongfully, that it would be created as soon as the user double-clicked the ZIP file in Explorer and viewed the directory listing within. However, that is not the case, and as long as the user stays within Explorer only, no folder is created.&lt;/p&gt;

&lt;p&gt;As an aside, I’ve noted that this also applies to nested folders within the ZIP archive - the user can browse them without triggering the &lt;code&gt;Temp1&lt;/code&gt; folder creation. However, if the ZIP archive contains &lt;em&gt;another&lt;/em&gt; ZIP archive and the user opens that with Explorer, a Temp1 folder is created for the &lt;em&gt;parent&lt;/em&gt; archive.&lt;/p&gt;

&lt;p&gt;The most common way a user will trigger the Temp1 folder’s creation, however, is by opening a file within the archive. This will create the folder and populate it with &lt;em&gt;only the file the user has actually opened&lt;/em&gt;. Subsequent files are copied to the folder as they are opened by the user, but it’s worth noting that the &lt;code&gt;Temp1&lt;/code&gt; folder &lt;em&gt;will not contain all files from the archive unless the user opens them all&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/12/explorer-temporary-zip-folders_2.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;These files are persistent regardless of whether the user closes them or opens other files. Interestingly, if the user makes changes to a file within the archive, by default the Save As dialogue box shows them the &lt;code&gt;Temp1&lt;/code&gt; folder, meaning an inattentive user might actually end up saving their edited copies to the same location (although this is unlikely, as they would have to give their file a different name).&lt;/p&gt;

&lt;p&gt;Once the &lt;code&gt;Temp1&lt;/code&gt; folder is created, it remains in place for as long as the user has &lt;em&gt;any&lt;/em&gt; ZIP archive open in Explorer. This is simple enough when working with one archive - the user closes the ZIP file and the &lt;code&gt;Temp1&lt;/code&gt; folder disappears. But if the user opens files within &lt;code&gt;a.zip&lt;/code&gt; and &lt;code&gt;b.zip&lt;/code&gt;, then closes &lt;code&gt;b.zip&lt;/code&gt;, the &lt;code&gt;Temp1&lt;/code&gt; folders for &lt;em&gt;both&lt;/em&gt; archives will remain until &lt;code&gt;a.zip&lt;/code&gt; is also closed in Explorer.&lt;/p&gt;

&lt;p&gt;It’s also worth noting here that the folder deletion doesn’t care about the files inside - it’ll happen whether they’re still open or not. The only scenario I could find where specific files matter is if the user saves an edited copy of a file (with a different name, since the originals are read-only) inside the &lt;code&gt;Temp1&lt;/code&gt; folder or moves a new file inside. If there’s a file that “doesn’t belong” at the time folder deletion is triggered, Explorer deletes the original files but leaves the directory and the “custom” files where they are.&lt;/p&gt;

&lt;p&gt;Another short note, this time on naming: The temporary folders almost always start &lt;code&gt;Temp1&lt;/code&gt;. In the previous example, there would be two folders named &lt;code&gt;Temp1_a.zip&lt;/code&gt; and &lt;code&gt;Temp1_b.zip&lt;/code&gt;. The only time the number increments is when dealing with two archives with &lt;em&gt;exactly the same name&lt;/em&gt;. If the user opens files within two archives, both called &lt;code&gt;a.zip&lt;/code&gt;, at the same time, Explorer creates &lt;code&gt;Temp1_a.zip&lt;/code&gt; and &lt;code&gt;Temp2_a.zip&lt;/code&gt;.&lt;/p&gt;

&lt;h3 id=&quot;summary&quot;&gt;Summary&lt;/h3&gt;

&lt;p&gt;Using the newfound knowledge from our experiments, we can draw up a rough guide to this behaviour and what your observations from log- and disk-based investigations might mean. As always, it’s best not to rely completely on this information and cross-reference with other artefacts wherever possible.&lt;/p&gt;

&lt;table&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;strong&gt;Observed&lt;/strong&gt;&lt;/td&gt;
      &lt;td&gt;&lt;strong&gt;Inferred&lt;/strong&gt;&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;code&gt;Temp1&lt;/code&gt; folder created&lt;/td&gt;
      &lt;td&gt;User opened a file within the ZIP archive with Explorer&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;File inside &lt;code&gt;Temp1&lt;/code&gt; folder&lt;/td&gt;
      &lt;td&gt;User opened this file from ZIP archive &lt;em&gt;OR&lt;/em&gt; manually wrote it to folder&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;code&gt;Temp1&lt;/code&gt; folder present&lt;/td&gt;
      &lt;td&gt;User still has at least one ZIP archive open &lt;em&gt;OR&lt;/em&gt; manually wrote a file inside&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;code&gt;Temp1&lt;/code&gt; folder deleted&lt;/td&gt;
      &lt;td&gt;User closed &lt;em&gt;all&lt;/em&gt; open ZIP archives in Explorer&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;code&gt;Temp1&lt;/code&gt; folder not created&lt;/td&gt;
      &lt;td&gt;User did not open any files from ZIP archive in Explorer&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;code&gt;Temp2&lt;/code&gt; folder created&lt;/td&gt;
      &lt;td&gt;User opened files inside two identically named ZIP files with Explorer&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;</content><author><name>mattcasmith</name></author><summary type="html">If I was to describe how often malware is downloaded within ZIP archives, “common” would be a huge understatement. A key artefact in these investigations is the temporary directory Windows creates when a user opens an archive in Explorer, but I recently realised I’d never actually run a proper test to see when the folder is created, when it is not, and when it is deleted. So to clear that up, here’s a quick blog post. ZIP archives on easy mode There was a time when a ZIP archive was an indecipherable box that Windows couldn’t see inside. To access the files within, you’d have to download third-party software, like a trial version of WinZip that would incessantly prompt you to buy a licence whenever you downloaded something from the internet. That all came to an end with - I think, although I may be slightly mistaken - Windows XP, when Microsoft added native capabilities within Windows Explorer to zip and unzip files. This is now a near-seamless experience, with the user able to open a ZIP archive as though it were a normal folder, as seen below. But what’s going on behind the scenes when this happens, and what artefacts does it leave behind for forensicators and responders? With so much malware downloaded from the internet and attached to emails inside ZIP archives, knowing where to look can be a powerful tool in tracing user actions. The amazing disappearing folder This focus of this post is the temporary directory that Explorer creates, quietly unzipping parts of the archive on demand in the background so that it can present the user with that seamless interface on the front end. It’s located within the user’s AppData Temp folder, under a path like this: C:\Users\user\AppData\Local\Temp\Temp1_zipname.zip\ But when exactly is the folder created? I previously assumed, wrongfully, that it would be created as soon as the user double-clicked the ZIP file in Explorer and viewed the directory listing within. However, that is not the case, and as long as the user stays within Explorer only, no folder is created. As an aside, I’ve noted that this also applies to nested folders within the ZIP archive - the user can browse them without triggering the Temp1 folder creation. However, if the ZIP archive contains another ZIP archive and the user opens that with Explorer, a Temp1 folder is created for the parent archive. The most common way a user will trigger the Temp1 folder’s creation, however, is by opening a file within the archive. This will create the folder and populate it with only the file the user has actually opened. Subsequent files are copied to the folder as they are opened by the user, but it’s worth noting that the Temp1 folder will not contain all files from the archive unless the user opens them all. These files are persistent regardless of whether the user closes them or opens other files. Interestingly, if the user makes changes to a file within the archive, by default the Save As dialogue box shows them the Temp1 folder, meaning an inattentive user might actually end up saving their edited copies to the same location (although this is unlikely, as they would have to give their file a different name). Once the Temp1 folder is created, it remains in place for as long as the user has any ZIP archive open in Explorer. This is simple enough when working with one archive - the user closes the ZIP file and the Temp1 folder disappears. But if the user opens files within a.zip and b.zip, then closes b.zip, the Temp1 folders for both archives will remain until a.zip is also closed in Explorer. It’s also worth noting here that the folder deletion doesn’t care about the files inside - it’ll happen whether they’re still open or not. The only scenario I could find where specific files matter is if the user saves an edited copy of a file (with a different name, since the originals are read-only) inside the Temp1 folder or moves a new file inside. If there’s a file that “doesn’t belong” at the time folder deletion is triggered, Explorer deletes the original files but leaves the directory and the “custom” files where they are. Another short note, this time on naming: The temporary folders almost always start Temp1. In the previous example, there would be two folders named Temp1_a.zip and Temp1_b.zip. The only time the number increments is when dealing with two archives with exactly the same name. If the user opens files within two archives, both called a.zip, at the same time, Explorer creates Temp1_a.zip and Temp2_a.zip. Summary Using the newfound knowledge from our experiments, we can draw up a rough guide to this behaviour and what your observations from log- and disk-based investigations might mean. As always, it’s best not to rely completely on this information and cross-reference with other artefacts wherever possible. Observed Inferred Temp1 folder created User opened a file within the ZIP archive with Explorer File inside Temp1 folder User opened this file from ZIP archive OR manually wrote it to folder Temp1 folder present User still has at least one ZIP archive open OR manually wrote a file inside Temp1 folder deleted User closed all open ZIP archives in Explorer Temp1 folder not created User did not open any files from ZIP archive in Explorer Temp2 folder created User opened files inside two identically named ZIP files with Explorer</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mattcasmith.net/wp-content/uploads/2022/12/explorer-temporary-zip-folders_0.png" /><media:content medium="image" url="https://mattcasmith.net/wp-content/uploads/2022/12/explorer-temporary-zip-folders_0.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Parsing login sessions from the Windows event log with PowerShell</title><link href="https://mattcasmith.net/2022/12/03/parsing-login-sessions-windows-security-event-log-powershell" rel="alternate" type="text/html" title="Parsing login sessions from the Windows event log with PowerShell" /><published>2022-12-03T00:00:00+00:00</published><updated>2022-12-03T00:00:00+00:00</updated><id>https://mattcasmith.net/2022/12/03/parsing-logon-sessions-windows-security-event-log-powershell</id><content type="html" xml:base="https://mattcasmith.net/2022/12/03/parsing-login-sessions-windows-security-event-log-powershell">&lt;p&gt;Faced with a day at home recovering from my most recent COVID-19 booster vaccine, I realised I hadn’t written anything more than a few lines of PowerShell in a while and decided to spend some time working on something interesting. The idea occurred to me to try to correlate Windows login sessions from the Security event log, and the Windows Logon Session EVTX Parser script is the result.&lt;/p&gt;

&lt;h3 id=&quot;contents&quot;&gt;Contents&lt;/h3&gt;

&lt;p&gt;1. &lt;a href=&quot;#introduction&quot;&gt;Introduction&lt;/a&gt;&lt;br /&gt;
          a. &lt;a href=&quot;#testing-and-limitations&quot;&gt;Testing and limitations&lt;/a&gt;&lt;br /&gt;
2. &lt;a href=&quot;#configuration&quot;&gt;Configuration&lt;/a&gt;&lt;br /&gt;
          a. &lt;a href=&quot;#verbosity&quot;&gt;Verbosity&lt;/a&gt;&lt;br /&gt;
3. &lt;a href=&quot;#reading-the-output&quot;&gt;Reading the output&lt;/a&gt;&lt;br /&gt;
4. &lt;a href=&quot;#download&quot;&gt;Download&lt;/a&gt;&lt;br /&gt;
5. &lt;a href=&quot;#future-development&quot;&gt;Future development&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;introduction&quot;&gt;Introduction&lt;/h3&gt;

&lt;p&gt;The Windows Logon Session EVTX Parser is a script that reads either a live or an exported Windows Security event log and produces a list of login sessions, including the login and logout times.&lt;/p&gt;

&lt;p&gt;Granted, Windows event logs - and particularly these events - are likely among the first logs you would onboard to a security information and event management (SIEM) tool in a corporate environment, but there may be situations where data isn’t so accessible - for example, when investigating a host that has not been onboarded to the central repository, or a personal device involved in a forensics case.&lt;/p&gt;

&lt;p style=&quot;float: left; width: 90%; padding: 5px; padding-left: 15px; padding-right: 5%; margin: 30px 0 10px 0; font-style: italic; border-left: 5px solid red; background: #ff9999; color: #333;&quot;&gt;This script is a learning/hobby project and some aspects may not follow best practices. While you're welcome to use it, you do so at your own risk. Make sure you take a backup of your files before trying it out, and don't go running it in production without proper checks.&lt;/p&gt;

&lt;p&gt;The Windows Security event log generates events with the ID &lt;a href=&quot;https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4624&quot; target=&quot;_blank&quot;&gt;&lt;code&gt;4624&lt;/code&gt;&lt;/a&gt; whenever a user logs in. These events contain a multitude of useful information, from the timestamp and username to the login type (e.g. local or remote) and - where relevant - the remote IP address used. When the user logs out, some combination of the events &lt;a href=&quot;https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4634&quot; target=&quot;_blank&quot;&gt;&lt;code&gt;4634&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4647&quot; target=&quot;_blank&quot;&gt;&lt;code&gt;4647&lt;/code&gt;&lt;/a&gt; is generated (see the links for more information on what and when).&lt;/p&gt;

&lt;p&gt;These events are linked together by a common field called &lt;code&gt;Logon ID&lt;/code&gt;, which is a pseudo-unique value specific to a single login session. But there’s a catch - Windows doesn’t make it very easy to use this data because the value is buried in each event’s &lt;code&gt;Message&lt;/code&gt; field. The Event Viewer doesn’t allow you to sort or filter based on it, and this gets crumpled into a single, hard-to-read cell in a CSV export.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/12/windows_logon_session_evtx_parser_0.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This is where the Windows Logon Session EVTX Parser comes in. Run the PowerShell script against a Windows Security event log and it will automatically find login and logout events, extract the relevant data from the &lt;code&gt;Message&lt;/code&gt; field, correlate events to identify login sessions, and present the findings in a neat table.&lt;/p&gt;

&lt;h4 id=&quot;testing-and-limitations&quot;&gt;Testing and limitations&lt;/h4&gt;

&lt;p&gt;Testing of the Windows Logon Session EVTX Parser is an ongoing process. The script is still a little rough around the edges (hopefully you’ll give me a pass on that one, given the circumstances under which I wrote it) and currently does not include error handling, so you may have to do some manual investigation if anything goes wrong. However, I have tested all the options (live and exported event logs, different verbosity levels, CSV and terminal-only output) and everything seems to be in working order.&lt;/p&gt;

&lt;h3 id=&quot;configuration&quot;&gt;Configuration&lt;/h3&gt;

&lt;p&gt;The Windows Logon Session EVTX Parser is relatively easy to configure by directly editing the variables within the script itself. There are three primary settings that can be adjusted to your needs.&lt;/p&gt;

&lt;table&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;strong&gt;Variable&lt;/strong&gt;&lt;/td&gt;
      &lt;td&gt;&lt;strong&gt;Example&lt;/strong&gt;&lt;/td&gt;
      &lt;td&gt;&lt;strong&gt;Purpose&lt;/strong&gt;&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;code&gt;$logFile&lt;/code&gt;&lt;/td&gt;
      &lt;td&gt;C:\temp\export.evtx&lt;/td&gt;
      &lt;td&gt;Either set to the path of an exported Windows Security event log EVTX file, or leave blank to use the live log (script must be run with administrator privileges if accessing live logs)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;code&gt;$verbosity&lt;/code&gt;&lt;/td&gt;
      &lt;td&gt;1&lt;/td&gt;
      &lt;td&gt;Must be set to one of the three verbosity settings (&lt;a href=&quot;#verbosity&quot;&gt;see below&lt;/a&gt;)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;code&gt;$outputFile&lt;/code&gt;&lt;/td&gt;
      &lt;td&gt;C:\temp\logon_sessions.csv&lt;/td&gt;
      &lt;td&gt;Either set a path to write a CSV file of logon sessions to, or leave blank to print logon sessions to the terminal only&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;Once the variables have been configured, the script can be run as a standard PowerShell script. If you wish to read the live event log, ensure that you are running the script from an administrator prompt.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;\&lt;span class=&quot;n&quot;&gt;windows&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;logon&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;session&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;evtx&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;parser&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;ps1&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;h4 id=&quot;verbosity&quot;&gt;Verbosity&lt;/h4&gt;

&lt;p&gt;The &lt;code&gt;$verbosity&lt;/code&gt; variable is perhaps the one that requires the most explanation. There are three possible levels of verbosity, and each has its own benefits and caveats, detailed below.&lt;/p&gt;

&lt;p&gt;Setting &lt;code&gt;$verbosity&lt;/code&gt; to &lt;code&gt;1&lt;/code&gt; will exclude &lt;code&gt;SYSTEM&lt;/code&gt; account logons and limit output to complete logon sessions (i.e. those with events for both the user logging in and logging out). This provides the most concise results, but will exclude any active sessions that do not yet have logout events, meaning it is best used in scenarios where its is known that the subject of the investigation is no longer active.&lt;/p&gt;

&lt;p&gt;Setting &lt;code&gt;$verbosity&lt;/code&gt; to &lt;code&gt;2&lt;/code&gt; will still exclude &lt;code&gt;SYSTEM&lt;/code&gt; account logons, but include a row in the output for each other login event, even if it does not have an associated logout event. This is likely the most useful verbosity level in most scenarios, as the results will include active sessions.&lt;/p&gt;

&lt;p&gt;Setting &lt;code&gt;$verbosity&lt;/code&gt; to &lt;code&gt;3&lt;/code&gt; will include a row in the output for all logins, including those for the &lt;code&gt;SYSTEM&lt;/code&gt; account and those with no associated logout event. This setting provides the most complete output, but includes a lot of noise from the &lt;code&gt;SYSTEM&lt;/code&gt; account that you’ll probably need to filter out yourself.&lt;/p&gt;

&lt;h3 id=&quot;reading-the-output&quot;&gt;Reading the output&lt;/h3&gt;

&lt;p&gt;Assuming you’ve populated the &lt;code&gt;$outputFile&lt;/code&gt; variable, the Windows Logon Session EVTX Parser will drop a CSV containing output of your selected verbosity at your chosen path. In the example below, the script returned complete login sessions only (&lt;code&gt;$verbosity = 1&lt;/code&gt;), and I have filtered out service account events.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/12/windows_logon_session_evtx_parser_1.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The timestamp and event ID columns are self-explanatory, but you might be wondering why I decided to include both &lt;code&gt;LoginUserName&lt;/code&gt; and &lt;code&gt;LogoutUserName&lt;/code&gt;. This is because when the user logs in using a Microsoft account, the login event shows the email address and the logout event shows the short/local username, so this can therefore be a useful way of correlating the two to determine which are related.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;LoginType&lt;/code&gt; shows &lt;a href=&quot;https://eventlogxp.com/blog/logon-type-what-does-it-mean/&quot; target=&quot;_blank&quot;&gt;the Windows logon type&lt;/a&gt;, which helps to understand &lt;em&gt;how&lt;/em&gt; the user logged on. In the example, I have highlighted a Type 3 (Network) logon to a shared folder from a virtual machine in orange, and a Type 4 (Batch) logon in green (this is actually the scheduled task for my backup utility, &lt;a href=&quot;/2021/01/01/backutil-windows-backup-utility&quot;&gt;Backutil&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;I’d recommend opening the output CSV in Excel and playing around with filters - particularly if you’ve run the script at the higher verbosity levels - but I’m sure most people reading this are already familiar.&lt;/p&gt;

&lt;h3 id=&quot;download&quot;&gt;Download&lt;/h3&gt;

&lt;p&gt;Use the link below to download the script. You’re free to run it as you wish - just please let me know if you encounter any bugs so I can work on fixing them in future releases!&lt;/p&gt;

&lt;p style=&quot;float: left; width: 90%; padding: 5px; padding-left: 15px; padding-right: 5%; margin: 30px 0 10px 0; font-style: italic; border-left: 5px solid red; background: #ff9999; color: #333;&quot;&gt;This script is a learning/hobby project and some aspects may not follow best practices. While you're welcome to use it, you do so at your own risk. Make sure you take a backup of your files before trying it out, and don't go running it in production without proper checks.&lt;/p&gt;

&lt;table&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;a href=&quot;https://github.com/mattcasmith/windows-logon-session-evtx-parser&quot;&gt;&lt;img src=&quot;/assets/images/download.png&quot; style=&quot;width: 50px&quot; /&gt;&lt;/a&gt;&lt;/td&gt;
      &lt;td&gt;&lt;a href=&quot;https://github.com/mattcasmith/windows-logon-session-evtx-parser&quot;&gt;Download WLSEP v0.1&lt;/a&gt;&lt;br /&gt;7KB, PS1&lt;/td&gt;
      &lt;td&gt;The PowerShell script is available to download directly from &lt;a target=&quot;_blank&quot; href=&quot;https://github.com/mattcasmith/windows-logon-session-evtx-parser&quot;&gt;the Windows Logon Session EVTX Parser GitHub repository&lt;/a&gt;.&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;h3 id=&quot;future-development&quot;&gt;Future development&lt;/h3&gt;

&lt;p&gt;As I mentioned earlier, the script is still rough in some areas, so when I have the time I would like to make a few improvements to tidy things up and improve functionality. These include:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;General housekeeping&lt;/strong&gt; - The script was written in a single day while I was feeling rather groggy, so there are almost certainly omissions, inefficiencies, and so on that I’ll find on review.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Error handling&lt;/strong&gt; - The script will currently run to completion while throwing PowerShell errors if something goes wrong. I’d like to add some proper error handling to deal with that more gracefully.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Additional functionality&lt;/strong&gt; - I would consider adding the option to include further events associated with Windows login sessions (e.g. failed logins) to the results at some point in future, and possibly the capability for the configuration variables to be provided via the command line.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Have you got more ideas for how the script could be improved? Or have you found bugs that I haven’t? Please &lt;a href=&quot;mailto:mattcasmith@protonmail.com&quot;&gt;send me an email&lt;/a&gt; to let me know so I can add them to the development backlog.&lt;/p&gt;

&lt;p&gt;If you’re interested in the project, check back regularly for new releases. I’ll also announce any updates on &lt;a target=&quot;_blank&quot; href=&quot;https://twitter.com/mattcasmith&quot;&gt;my Twitter account&lt;/a&gt;, and may add some form of banner to &lt;a href=&quot;https://mattcasmith.net&quot;&gt;my site’s homepage&lt;/a&gt;.&lt;/p&gt;</content><author><name>mattcasmith</name></author><summary type="html">Faced with a day at home recovering from my most recent COVID-19 booster vaccine, I realised I hadn’t written anything more than a few lines of PowerShell in a while and decided to spend some time working on something interesting. The idea occurred to me to try to correlate Windows login sessions from the Security event log, and the Windows Logon Session EVTX Parser script is the result. Contents 1. Introduction           a. Testing and limitations 2. Configuration           a. Verbosity 3. Reading the output 4. Download 5. Future development Introduction The Windows Logon Session EVTX Parser is a script that reads either a live or an exported Windows Security event log and produces a list of login sessions, including the login and logout times. Granted, Windows event logs - and particularly these events - are likely among the first logs you would onboard to a security information and event management (SIEM) tool in a corporate environment, but there may be situations where data isn’t so accessible - for example, when investigating a host that has not been onboarded to the central repository, or a personal device involved in a forensics case. This script is a learning/hobby project and some aspects may not follow best practices. While you're welcome to use it, you do so at your own risk. Make sure you take a backup of your files before trying it out, and don't go running it in production without proper checks. The Windows Security event log generates events with the ID 4624 whenever a user logs in. These events contain a multitude of useful information, from the timestamp and username to the login type (e.g. local or remote) and - where relevant - the remote IP address used. When the user logs out, some combination of the events 4634 and 4647 is generated (see the links for more information on what and when). These events are linked together by a common field called Logon ID, which is a pseudo-unique value specific to a single login session. But there’s a catch - Windows doesn’t make it very easy to use this data because the value is buried in each event’s Message field. The Event Viewer doesn’t allow you to sort or filter based on it, and this gets crumpled into a single, hard-to-read cell in a CSV export. This is where the Windows Logon Session EVTX Parser comes in. Run the PowerShell script against a Windows Security event log and it will automatically find login and logout events, extract the relevant data from the Message field, correlate events to identify login sessions, and present the findings in a neat table. Testing and limitations Testing of the Windows Logon Session EVTX Parser is an ongoing process. The script is still a little rough around the edges (hopefully you’ll give me a pass on that one, given the circumstances under which I wrote it) and currently does not include error handling, so you may have to do some manual investigation if anything goes wrong. However, I have tested all the options (live and exported event logs, different verbosity levels, CSV and terminal-only output) and everything seems to be in working order. Configuration The Windows Logon Session EVTX Parser is relatively easy to configure by directly editing the variables within the script itself. There are three primary settings that can be adjusted to your needs. Variable Example Purpose $logFile C:\temp\export.evtx Either set to the path of an exported Windows Security event log EVTX file, or leave blank to use the live log (script must be run with administrator privileges if accessing live logs) $verbosity 1 Must be set to one of the three verbosity settings (see below) $outputFile C:\temp\logon_sessions.csv Either set a path to write a CSV file of logon sessions to, or leave blank to print logon sessions to the terminal only Once the variables have been configured, the script can be run as a standard PowerShell script. If you wish to read the live event log, ensure that you are running the script from an administrator prompt. .\windows-logon-session-evtx-parser.ps1 Verbosity The $verbosity variable is perhaps the one that requires the most explanation. There are three possible levels of verbosity, and each has its own benefits and caveats, detailed below. Setting $verbosity to 1 will exclude SYSTEM account logons and limit output to complete logon sessions (i.e. those with events for both the user logging in and logging out). This provides the most concise results, but will exclude any active sessions that do not yet have logout events, meaning it is best used in scenarios where its is known that the subject of the investigation is no longer active. Setting $verbosity to 2 will still exclude SYSTEM account logons, but include a row in the output for each other login event, even if it does not have an associated logout event. This is likely the most useful verbosity level in most scenarios, as the results will include active sessions. Setting $verbosity to 3 will include a row in the output for all logins, including those for the SYSTEM account and those with no associated logout event. This setting provides the most complete output, but includes a lot of noise from the SYSTEM account that you’ll probably need to filter out yourself. Reading the output Assuming you’ve populated the $outputFile variable, the Windows Logon Session EVTX Parser will drop a CSV containing output of your selected verbosity at your chosen path. In the example below, the script returned complete login sessions only ($verbosity = 1), and I have filtered out service account events. The timestamp and event ID columns are self-explanatory, but you might be wondering why I decided to include both LoginUserName and LogoutUserName. This is because when the user logs in using a Microsoft account, the login event shows the email address and the logout event shows the short/local username, so this can therefore be a useful way of correlating the two to determine which are related. LoginType shows the Windows logon type, which helps to understand how the user logged on. In the example, I have highlighted a Type 3 (Network) logon to a shared folder from a virtual machine in orange, and a Type 4 (Batch) logon in green (this is actually the scheduled task for my backup utility, Backutil). I’d recommend opening the output CSV in Excel and playing around with filters - particularly if you’ve run the script at the higher verbosity levels - but I’m sure most people reading this are already familiar. Download Use the link below to download the script. You’re free to run it as you wish - just please let me know if you encounter any bugs so I can work on fixing them in future releases! This script is a learning/hobby project and some aspects may not follow best practices. While you're welcome to use it, you do so at your own risk. Make sure you take a backup of your files before trying it out, and don't go running it in production without proper checks. Download WLSEP v0.17KB, PS1 The PowerShell script is available to download directly from the Windows Logon Session EVTX Parser GitHub repository. Future development As I mentioned earlier, the script is still rough in some areas, so when I have the time I would like to make a few improvements to tidy things up and improve functionality. These include: General housekeeping - The script was written in a single day while I was feeling rather groggy, so there are almost certainly omissions, inefficiencies, and so on that I’ll find on review. Error handling - The script will currently run to completion while throwing PowerShell errors if something goes wrong. I’d like to add some proper error handling to deal with that more gracefully. Additional functionality - I would consider adding the option to include further events associated with Windows login sessions (e.g. failed logins) to the results at some point in future, and possibly the capability for the configuration variables to be provided via the command line. Have you got more ideas for how the script could be improved? Or have you found bugs that I haven’t? Please send me an email to let me know so I can add them to the development backlog. If you’re interested in the project, check back regularly for new releases. I’ll also announce any updates on my Twitter account, and may add some form of banner to my site’s homepage.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mattcasmith.net/wp-content/uploads/2022/12/windows_logon_session_evtx_parser_0.png" /><media:content medium="image" url="https://mattcasmith.net/wp-content/uploads/2022/12/windows_logon_session_evtx_parser_0.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Using Tkinter to build simple GUIs for Python apps</title><link href="https://mattcasmith.net/2022/08/21/using-tkinter-gui-python-apps" rel="alternate" type="text/html" title="Using Tkinter to build simple GUIs for Python apps" /><published>2022-08-21T01:00:00+01:00</published><updated>2022-08-21T01:00:00+01:00</updated><id>https://mattcasmith.net/2022/08/21/using-tkinter-gui-python-apps</id><content type="html" xml:base="https://mattcasmith.net/2022/08/21/using-tkinter-gui-python-apps">&lt;p&gt;I’ve written many Python scripts and apps in the past, including my Windows backup utility &lt;a href=&quot;/2021/01/01/backutil-windows-backup-utility&quot;&gt;Backutil&lt;/a&gt;, which is probably the my most complex project to date. But I’ve always designed these to run in the background or on the command line, and haven’t ever tried to build an app with a graphical user interface (GUI).&lt;/p&gt;

&lt;p&gt;I was recently studying for a few &lt;a href=&quot;/category/cyber-security&quot;&gt;cyber security&lt;/a&gt; certifications and realised it would be useful to have an app for displaying flashcards. There are probably a million solutions for this online already, but I thought it would be a good opportunity to learn how to add a GUI to control some simple Python functions. So I took to Google, looked up some tutorials, and cobbled something together on a Saturday afternoon.&lt;/p&gt;

&lt;p&gt;I’ll share the full code in future via GitHub and another blog post (there are still some more features I want to add), but for now I’ll focus on the GUI-related elements and how they connect to everything else.&lt;/p&gt;

&lt;h3 id=&quot;introducting-tkinter&quot;&gt;Introducting Tkinter&lt;/h3&gt;

&lt;p&gt;&lt;a href=&quot;https://docs.python.org/3/library/tkinter.html&quot; target=&quot;_blank&quot;&gt;Tkinter&lt;/a&gt; is a Python library that allows you to work with the Tcl/Tk GUI toolkit to create applications that will work across Windows, macOS, and Linux. You can visit the link to review the full documentation, but I’ll cover some of the basics I learnt and how I used them for my flashcards app in this post.&lt;/p&gt;

&lt;p&gt;As with any Python extension, the first step is to import the library.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;kn&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;tkinter&lt;/span&gt; &lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt;
&lt;span class=&quot;kn&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;tkinter&lt;/span&gt; &lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;ttk&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;To understand what comes next, it’s probably better to see what the end result looks like first, so here’s the very simple GUI that we’ll be constructing in the following code segments for the flashcards app.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/08/tkinter-python_0.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;As you can see, we have a main area that displays the content of the current flashcard, a button to flip the card to the other side, previous/next buttons and a counter to show the current position, and a button to reshuffle the deck into a new random order. There are also some keyboard shortcuts so the user doesn’t always need to navigate with the mouse. In the following sections, I’ll go over how each of these work.&lt;/p&gt;

&lt;h3 id=&quot;setting-up-the-window-and-grid&quot;&gt;Setting up the window and grid&lt;/h3&gt;

&lt;p&gt;We need a canvas before we can do any painting, and in this case the canvas is the application’s main window - also known as &lt;code&gt;root&lt;/code&gt;. We can set this up with a few simple lines of code.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;n&quot;&gt;root&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Tk&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;root&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;title&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Flashcards!&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;root&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;geometry&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;400x400&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;You can probably already glean the basics just by reading this, but with &lt;code&gt;title&lt;/code&gt; we have set the window title to &lt;code&gt;Flashcards!&lt;/code&gt;, and with &lt;code&gt;geometry&lt;/code&gt; we have set the default window size to 400 by 400 pixels.&lt;/p&gt;

&lt;p&gt;Before we can place individual elements in our window, we need to establish a grid, which we’ll call &lt;code&gt;mainframe&lt;/code&gt;. The following lines create this frame and centre it with the compass-point &lt;code&gt;sticky&lt;/code&gt; attributes.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;n&quot;&gt;mainframe&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;ttk&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Frame&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;root&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;width&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;400&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;height&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;400&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;mainframe&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;grid&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;column&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;row&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sticky&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;N&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;W&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;E&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;S&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;With the grid itself set up, we can now apply attributes to the individual rows and columns. My basic GUI will require four rows, but only one of them needs any special configuration. I’m using &lt;code&gt;weight&lt;/code&gt; and &lt;code&gt;minsize&lt;/code&gt; on the second row to reserve plenty of space for the content of the cards.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;n&quot;&gt;mainframe&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;rowconfigure&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;weight&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;8&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;minsize&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;295&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;And that’s our grid set up and formatted! To help you to visualise where we’re adding each button and label in the next section, here’s roughly how the grid aligns to the finished GUI.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/08/tkinter-python_1.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Notice how the tweaks we made to the second row ensure there’s plenty of space in the middle in case the user provides lengthy content for their flashcards. You’ll probably also spot that the main card content and the Flip button don’t respect the column layout - we’ll see how to achieve this in a moment.&lt;/p&gt;

&lt;h3 id=&quot;buttons-labels-and-string-variables&quot;&gt;Buttons, labels, and string variables&lt;/h3&gt;

&lt;p&gt;Now we can get to placing the window contents - but we still have one more important job to do first. The text in the main card content section and the counter in the top right will need to change based on the user’s input, and to be able to update them we must establish them as &lt;code&gt;StringVar()&lt;/code&gt; variables.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;n&quot;&gt;card_content&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;StringVar&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;card_number&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;StringVar&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;Then we create each button and label individually and assign their attributes.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;n&quot;&gt;ttk&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Button&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;mainframe&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Flip&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;command&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;flip_card&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;grid&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;column&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;row&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;columnspan&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sticky&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;W&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;E&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;S&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;ttk&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Button&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;mainframe&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&amp;lt;&amp;lt;&amp;lt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;command&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;prev_card&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;width&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;15&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;grid&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;column&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;row&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sticky&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;W&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;S&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;ttk&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Button&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;mainframe&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Shuffle&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;command&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;randomise_data&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;width&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;15&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;grid&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;column&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;row&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sticky&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;S&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;ttk&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Button&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;mainframe&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&amp;gt;&amp;gt;&amp;gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;command&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;next_card&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;width&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;15&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;grid&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;column&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;row&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sticky&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;E&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;S&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;ttk&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Label&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;mainframe&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;font&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Verdana&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;textvariable&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;card_content&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;wraplength&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;390&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;anchor&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;center&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;width&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;38&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;grid&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;column&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;row&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;columnspan&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;ttk&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Label&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;mainframe&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;textvariable&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;card_number&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;anchor&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;e&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;width&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;15&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;grid&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;column&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;row&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;ttk&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Label&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;mainframe&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;MattCASmith.net&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;anchor&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;w&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;width&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;17&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;grid&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;column&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;row&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;In the brackets after each &lt;code&gt;Button&lt;/code&gt; or &lt;code&gt;Label&lt;/code&gt; is created, we first state which element it should be placed within (&lt;code&gt;mainframe&lt;/code&gt;) and then provide various configurations. There are three that are worth explaining:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;code&gt;textvariable&lt;/code&gt; - used to assign the two &lt;code&gt;StringVar()&lt;/code&gt; variables we established to their respective labels. This is necessary for dynamic text. If the text is static, &lt;code&gt;text&lt;/code&gt; can be used instead&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;code&gt;anchor&lt;/code&gt; - sets the alignment of the label text based on compass point values&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;code&gt;command&lt;/code&gt; - links buttons to the Python functions that must be executed when they are clicked. For example, clicking the Shuffle button will execute the &lt;code&gt;randomise_data&lt;/code&gt; function&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In the brackets after &lt;code&gt;grid&lt;/code&gt; we establish the placement of each element in the grid we set up earlier. The &lt;code&gt;column&lt;/code&gt; and &lt;code&gt;row&lt;/code&gt; numbers assign a cell to the element, and &lt;code&gt;sticky&lt;/code&gt; sets its alignment within that cell. An interesting attribute here is &lt;code&gt;columnspan&lt;/code&gt;, which allows an element to sit across multiple columns - as is the case here with the main content and the Shuffle button.&lt;/p&gt;

&lt;h3 id=&quot;keyboard-shortcuts&quot;&gt;Keyboard shortcuts&lt;/h3&gt;

&lt;p&gt;With our GUI mostly in place, we have an issue. When revising using a flashcards app, you’d want to be able to cycle quickly between the cards and flip them instantly to see whether you’re right or wrong. But at the moment our user needs to click on the corresponding buttons each time they want to change the view. So let’s fix that with some keyboard shortcuts so they can study at speed.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;n&quot;&gt;root&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;bind&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;'&amp;lt;Left&amp;gt;'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;prev_card&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;root&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;bind&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;'&amp;lt;Right&amp;gt;'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;next_card&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;root&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;bind&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;'&amp;lt;Return&amp;gt;'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;flip_card&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;This should be fairly self-explanatory, but essentially what each line achieves is to bind a Python function (&lt;code&gt;prev_card&lt;/code&gt;, &lt;code&gt;next_card&lt;/code&gt;, &lt;code&gt;flip_card&lt;/code&gt;) to a key (the left and right arrows and return, respectively), reducing the work required to cycle through cards and providing a much smoother user experience.&lt;/p&gt;

&lt;h3 id=&quot;drawing-the-window&quot;&gt;Drawing the window&lt;/h3&gt;

&lt;p&gt;There’s just a little more admin to do before our application will run properly. Firstly, let’s add some padding to each cell in the grid to space out our buttons and labels a little.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;child&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;mainframe&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;winfo_children&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;():&lt;/span&gt; 
    &lt;span class=&quot;n&quot;&gt;child&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;grid_configure&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;padx&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;5&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;pady&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;5&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;Finally, we use &lt;code&gt;mainloop()&lt;/code&gt; to run our application and draw the window. When execution reaches this line, the window will be drawn and the app will essentially sit idle awaiting user input, so it’s important this we did the necessary legwork beforehand to make sure everything is in place and presentable.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;n&quot;&gt;root&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;mainloop&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;And there you have it - this code (and the functions required to provide the actual functionality) should be all you need to create a very simple GUI application using Python. Flashcards! is currently at the minimum viable product stage - it’s perfectly usable as a study tool, but still rough round the edges. Once I’ve had a chance to spruce it up a bit I’ll be sure to share the full project code. Watch this space!&lt;/p&gt;</content><author><name>mattcasmith</name></author><summary type="html">I’ve written many Python scripts and apps in the past, including my Windows backup utility Backutil, which is probably the my most complex project to date. But I’ve always designed these to run in the background or on the command line, and haven’t ever tried to build an app with a graphical user interface (GUI). I was recently studying for a few cyber security certifications and realised it would be useful to have an app for displaying flashcards. There are probably a million solutions for this online already, but I thought it would be a good opportunity to learn how to add a GUI to control some simple Python functions. So I took to Google, looked up some tutorials, and cobbled something together on a Saturday afternoon. I’ll share the full code in future via GitHub and another blog post (there are still some more features I want to add), but for now I’ll focus on the GUI-related elements and how they connect to everything else. Introducting Tkinter Tkinter is a Python library that allows you to work with the Tcl/Tk GUI toolkit to create applications that will work across Windows, macOS, and Linux. You can visit the link to review the full documentation, but I’ll cover some of the basics I learnt and how I used them for my flashcards app in this post. As with any Python extension, the first step is to import the library. from tkinter import * from tkinter import ttk To understand what comes next, it’s probably better to see what the end result looks like first, so here’s the very simple GUI that we’ll be constructing in the following code segments for the flashcards app. As you can see, we have a main area that displays the content of the current flashcard, a button to flip the card to the other side, previous/next buttons and a counter to show the current position, and a button to reshuffle the deck into a new random order. There are also some keyboard shortcuts so the user doesn’t always need to navigate with the mouse. In the following sections, I’ll go over how each of these work. Setting up the window and grid We need a canvas before we can do any painting, and in this case the canvas is the application’s main window - also known as root. We can set this up with a few simple lines of code. root = Tk() root.title(&quot;Flashcards!&quot;) root.geometry(&quot;400x400&quot;) You can probably already glean the basics just by reading this, but with title we have set the window title to Flashcards!, and with geometry we have set the default window size to 400 by 400 pixels. Before we can place individual elements in our window, we need to establish a grid, which we’ll call mainframe. The following lines create this frame and centre it with the compass-point sticky attributes. mainframe = ttk.Frame(root, width=400, height=400) mainframe.grid(column=0, row=0, sticky=(N, W, E, S)) With the grid itself set up, we can now apply attributes to the individual rows and columns. My basic GUI will require four rows, but only one of them needs any special configuration. I’m using weight and minsize on the second row to reserve plenty of space for the content of the cards. mainframe.rowconfigure(1, weight=8, minsize=295) And that’s our grid set up and formatted! To help you to visualise where we’re adding each button and label in the next section, here’s roughly how the grid aligns to the finished GUI. Notice how the tweaks we made to the second row ensure there’s plenty of space in the middle in case the user provides lengthy content for their flashcards. You’ll probably also spot that the main card content and the Flip button don’t respect the column layout - we’ll see how to achieve this in a moment. Buttons, labels, and string variables Now we can get to placing the window contents - but we still have one more important job to do first. The text in the main card content section and the counter in the top right will need to change based on the user’s input, and to be able to update them we must establish them as StringVar() variables. card_content = StringVar() card_number = StringVar() Then we create each button and label individually and assign their attributes. ttk.Button(mainframe, text=&quot;Flip&quot;, command=flip_card).grid(column=1, row=2, columnspan=3, sticky=(W,E,S)) ttk.Button(mainframe, text=&quot;&amp;lt;&amp;lt;&amp;lt;&quot;, command=prev_card, width=15).grid(column=1, row=3, sticky=(W,S)) ttk.Button(mainframe, text=&quot;Shuffle&quot;, command=randomise_data, width=15).grid(column=2, row=3, sticky=S) ttk.Button(mainframe, text=&quot;&amp;gt;&amp;gt;&amp;gt;&quot;, command=next_card, width=15).grid(column=3, row=3, sticky=(E,S)) ttk.Label(mainframe, font=&quot;Verdana&quot;, textvariable=card_content, wraplength=390, anchor=&quot;center&quot;, width=38).grid(column=1, row=1, columnspan=3) ttk.Label(mainframe, textvariable=card_number, anchor=&quot;e&quot;, width=15).grid(column=3, row=0) ttk.Label(mainframe, text=&quot;MattCASmith.net&quot;, anchor=&quot;w&quot;, width=17).grid(column=1, row=0) In the brackets after each Button or Label is created, we first state which element it should be placed within (mainframe) and then provide various configurations. There are three that are worth explaining: textvariable - used to assign the two StringVar() variables we established to their respective labels. This is necessary for dynamic text. If the text is static, text can be used instead anchor - sets the alignment of the label text based on compass point values command - links buttons to the Python functions that must be executed when they are clicked. For example, clicking the Shuffle button will execute the randomise_data function In the brackets after grid we establish the placement of each element in the grid we set up earlier. The column and row numbers assign a cell to the element, and sticky sets its alignment within that cell. An interesting attribute here is columnspan, which allows an element to sit across multiple columns - as is the case here with the main content and the Shuffle button. Keyboard shortcuts With our GUI mostly in place, we have an issue. When revising using a flashcards app, you’d want to be able to cycle quickly between the cards and flip them instantly to see whether you’re right or wrong. But at the moment our user needs to click on the corresponding buttons each time they want to change the view. So let’s fix that with some keyboard shortcuts so they can study at speed. root.bind('&amp;lt;Left&amp;gt;', prev_card) root.bind('&amp;lt;Right&amp;gt;', next_card) root.bind('&amp;lt;Return&amp;gt;', flip_card) This should be fairly self-explanatory, but essentially what each line achieves is to bind a Python function (prev_card, next_card, flip_card) to a key (the left and right arrows and return, respectively), reducing the work required to cycle through cards and providing a much smoother user experience. Drawing the window There’s just a little more admin to do before our application will run properly. Firstly, let’s add some padding to each cell in the grid to space out our buttons and labels a little. for child in mainframe.winfo_children(): child.grid_configure(padx=5, pady=5) Finally, we use mainloop() to run our application and draw the window. When execution reaches this line, the window will be drawn and the app will essentially sit idle awaiting user input, so it’s important this we did the necessary legwork beforehand to make sure everything is in place and presentable. root.mainloop() And there you have it - this code (and the functions required to provide the actual functionality) should be all you need to create a very simple GUI application using Python. Flashcards! is currently at the minimum viable product stage - it’s perfectly usable as a study tool, but still rough round the edges. Once I’ve had a chance to spruce it up a bit I’ll be sure to share the full project code. Watch this space!</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mattcasmith.net/wp-content/uploads/2022/08/tkinter-python_0.png" /><media:content medium="image" url="https://mattcasmith.net/wp-content/uploads/2022/08/tkinter-python_0.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Linux .bash_history: Basics, behaviours, and forensics</title><link href="https://mattcasmith.net/2022/02/22/bash-history-basics-behaviours-forensics" rel="alternate" type="text/html" title="Linux .bash&amp;#95;history&amp;#58; Basics, behaviours, and forensics" /><published>2022-02-22T00:00:00+00:00</published><updated>2022-02-22T00:00:00+00:00</updated><id>https://mattcasmith.net/2022/02/22/bash-history-basics-behaviours-forensics</id><content type="html" xml:base="https://mattcasmith.net/2022/02/22/bash-history-basics-behaviours-forensics">&lt;p&gt;During any incident investigation on a Linux system, one of the most valuable things for responders and forensicators to establish is which commands were run. This is key to finding out what an attacker or malicious user was attempting to do, and what remediation activities are required.&lt;/p&gt;

&lt;p&gt;The &lt;code&gt;.bash_history&lt;/code&gt; file, located in each user’s home directory, is usually the investigator’s first stop for this information. This file contains a list of Bash commands recently run by the user and may appear relatively simple at first glance, but there are many ins and outs to its behaviours.&lt;/p&gt;

&lt;p&gt;Most importantly, commands entered in a running Bash terminal are stored in memory and are only written to the file when the terminal is closed. As you can imagine, this means many caveats can be applied to the file’s contents, and in this post I aim to cover some of the most common scenarios, how they affect &lt;code&gt;.bash_history&lt;/code&gt;, and alternatives when it does not contain the activity you are looking for.&lt;/p&gt;

&lt;h3 id=&quot;contents&quot;&gt;Contents&lt;/h3&gt;

&lt;p&gt;1. &lt;a href=&quot;#bash_history-behaviours&quot;&gt;&lt;code&gt;.bash_history&lt;/code&gt; behaviours&lt;/a&gt;&lt;br /&gt;
          a. &lt;a href=&quot;#first-a-note-on-the-histcontrol-variable&quot;&gt;The &lt;code&gt;$HISTCONTROL&lt;/code&gt; variable&lt;/a&gt;&lt;br /&gt;
          b. &lt;a href=&quot;#the-history-command-outputs-a-combined-history-from-bash_history-and-memory&quot;&gt;The &lt;code&gt;history&lt;/code&gt; command&lt;/a&gt;&lt;br /&gt;
          c. &lt;a href=&quot;#commands-are-written-to-bash_history-in-the-order-terminals-are-closed-not-the-order-they-are-run&quot;&gt;Terminal windows and command order&lt;/a&gt;&lt;br /&gt;
          d. &lt;a href=&quot;#commands-run-with-logical-and--appear-as-a-single-entry-in-bash_history&quot;&gt;Logical AND (&lt;code&gt;&amp;amp;&amp;amp;&lt;/code&gt;)&lt;/a&gt;&lt;br /&gt;
          e. &lt;a href=&quot;#commands-run-within-scripts-are-not-reflected-in-bash_history&quot;&gt;Commands within scripts&lt;/a&gt;&lt;br /&gt;
          f. &lt;a href=&quot;#if-a-terminal-is-stopped-with-the-kill-command-commands-are-still-written-to-bash_history&quot;&gt;The &lt;code&gt;kill&lt;/code&gt; command&lt;/a&gt;&lt;br /&gt;
          g. &lt;a href=&quot;#however-if-the--sigkill-switch-is-used-commands-are-not-written-to-bash_history&quot;&gt;The &lt;code&gt;-SIGKILL&lt;/code&gt; switch&lt;/a&gt;&lt;br /&gt;
          h. &lt;a href=&quot;#ssh-sessions-behave-in-a-similar-way-to-standard-terminals&quot;&gt;SSH sessions&lt;/a&gt;&lt;br /&gt;
2. &lt;a href=&quot;#finding-bash-commands-in-memory&quot;&gt;Finding Bash commands in memory&lt;/a&gt;&lt;br /&gt;
3. &lt;a href=&quot;#further-reading&quot;&gt;Further reading&lt;/a&gt;&lt;br /&gt;&lt;/p&gt;

&lt;h3 id=&quot;bash_history-behaviours&quot;&gt;&lt;code&gt;.bash_history&lt;/code&gt; behaviours&lt;/h3&gt;

&lt;p&gt;To test exactly when commands are and aren’t recorded to the user’s &lt;code&gt;.bash_history&lt;/code&gt; file, I ran a series of tests covering common scenarios in which commands might be run. All of these tests were run on a clean installation of CentOS. Here’s what I found…&lt;/p&gt;

&lt;h4 id=&quot;first-a-note-on-the-histcontrol-variable&quot;&gt;First, a note on the &lt;code&gt;$HISTCONTROL&lt;/code&gt; variable…&lt;/h4&gt;

&lt;p&gt;It is important to check the contents of the &lt;code&gt;$HISTCONTROL&lt;/code&gt; variable on the subject system (after taking images and preserving evidence, so you’re not overwriting &lt;code&gt;.bash_history&lt;/code&gt; with your own commands) as this has a potentially significant bearing on which commands will be written to the file.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;echo $HISTCONTROL&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The command above will return one of the following strings. The table below details which commands &lt;code&gt;.bash_history&lt;/code&gt; will ignore when each of the possible values is present.&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th style=&quot;text-align: left&quot;&gt;$HISTCONTROL value&lt;/th&gt;
      &lt;th style=&quot;text-align: left&quot;&gt;Description&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;ignorespace&lt;/td&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;Excludes commands with a preceding space&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;ignoredups&lt;/td&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;Excludes subsequent duplicate commands&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;ignorespace:ignoredups&lt;/td&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;Excludes both commands with a preceding space and subsequent duplicates&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;ignoreboth&lt;/td&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;Excludes both commands with a preceding space and subsequent duplicates&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;Therefore if the system has one of these values in the &lt;code&gt;$HISTCONTROL&lt;/code&gt; variable, there may be commands missing from &lt;code&gt;.bash_history&lt;/code&gt;. If it is set to &lt;code&gt;ignorespace&lt;/code&gt; and the attacker is aware, they could even just slip in a space before each of their commands and write nothing to the history file at all!&lt;/p&gt;

&lt;p&gt;The CentOS system where I performed the testing for this post had &lt;code&gt;$HISTCONTROL&lt;/code&gt; set to &lt;code&gt;ignoredups&lt;/code&gt;. However, I also performed checks across other Linux distros I had available (namely Kali, Tsurugi, and Windows Subsystem for Linux) and it was set to &lt;code&gt;ignoreboth&lt;/code&gt; for them all. So it’s important to check what you’re dealing with on the particular system you’re investigating.&lt;/p&gt;

&lt;p&gt;It’s also trivial to change the contents of the &lt;code&gt;$HISTCONTROL&lt;/code&gt; variable to exclude more commands from &lt;code&gt;.bash_history&lt;/code&gt;, so watch out for evidence it has changed, which will look something like the following:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;export HISTCONTROL=ignoreboth&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h4 id=&quot;the-history-command-outputs-a-combined-history-from-bash_history-and-memory&quot;&gt;The &lt;code&gt;history&lt;/code&gt; command outputs a combined history from &lt;code&gt;.bash_history&lt;/code&gt; and memory.&lt;/h4&gt;

&lt;p&gt;On a live Linux system, you might run the &lt;code&gt;history&lt;/code&gt; command in a terminal window to review recently executed commands. The output of this command combines both the contents of &lt;code&gt;.bash_history&lt;/code&gt; and any commands held in memory from the current session.&lt;/p&gt;

&lt;p&gt;In the example below, we can see that the test command &lt;code&gt;doesthiscommandshowinhistory?&lt;/code&gt; is returned by the &lt;code&gt;history&lt;/code&gt; command on the left, but has not yet been written to disk as shown on the right.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/02/bash-history_0.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Bear in mind that &lt;code&gt;history&lt;/code&gt; will show commands from the memory of only the current Bash terminal - it won’t have access to those belonging to other processes. Since you won’t have access to the attacker’s terminal (unless the incident response team directly interrupted a hands-on-keyboard session), this means it won’t usually be much more useful than &lt;code&gt;.bash_history&lt;/code&gt; itself.&lt;/p&gt;

&lt;h4 id=&quot;commands-are-written-to-bash_history-in-the-order-terminals-are-closed-not-the-order-they-are-run&quot;&gt;Commands are written to &lt;code&gt;.bash_history&lt;/code&gt; in the order terminals are closed, not the order they are run.&lt;/h4&gt;

&lt;p&gt;For this test, I opened two Bash terminals and ran example commands containing text stating which terminal they belonged to and in which order they were run. As I closed the first terminal before the second, you can see that both of its commands appear first in &lt;code&gt;.bash_history&lt;/code&gt;, even though some of the second terminal’s commands were run first. It is the order the Bash processes end that matters.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/02/bash-history_1.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;As was discussed before, note that if we had run the &lt;code&gt;history&lt;/code&gt; command in one of these terminals, it would only have returned its own commands because each process has its own space in memory.&lt;/p&gt;

&lt;h4 id=&quot;commands-run-with-logical-and--appear-as-a-single-entry-in-bash_history&quot;&gt;Commands run with logical AND (&lt;code&gt;&amp;amp;&amp;amp;&lt;/code&gt;) appear as a single entry in &lt;code&gt;.bash_history&lt;/code&gt;.&lt;/h4&gt;

&lt;p&gt;If we string several commands together using &lt;code&gt;&amp;amp;&amp;amp;&lt;/code&gt; - a technique often used to save time (or in less innocent circumstances to avoid an attacker having to submit commands multiple times and risk attracting unwanted attention) - they will still only appear as one line in &lt;code&gt;.bash_history&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/02/bash-history_1.1.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This is an important detail as investigators and forensicators, as it preserves the original context in which these commands were run. If they were run back-to-back in this manner, this invites the question: “Why?”&lt;/p&gt;

&lt;h4 id=&quot;commands-run-within-scripts-are-not-reflected-in-bash_history&quot;&gt;Commands run within scripts are not reflected in &lt;code&gt;.bash_history&lt;/code&gt;.&lt;/h4&gt;

&lt;p&gt;To see whether commands in scripts are added to &lt;code&gt;.bash_history&lt;/code&gt; or not, I created a script called &lt;code&gt;folder_test.sh&lt;/code&gt; that would create a new folder, navigate into it, list the contents, navigate back up a level, and then delete the folder. You can see its contents in the left-hand window below.&lt;/p&gt;

&lt;p&gt;On the right, I ran the script with &lt;code&gt;./folder_test.sh&lt;/code&gt;. We know it executed successfully because the contents (i.e. nothing) are printed to the terminal. However, when we check the output of the &lt;code&gt;history&lt;/code&gt; command we can see that although the command that &lt;em&gt;ran&lt;/em&gt; the script was recorded, commands run &lt;em&gt;within&lt;/em&gt; the script were not - an important distinction to remember when investigating Linux systems.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/02/bash-history_1.2.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This means that our attacker could potentially download or create a script named something innocuous like &lt;code&gt;file_cleanup.sh&lt;/code&gt;, execute it, and we would be none the wiser as to what it did based on the contents of &lt;code&gt;.bash_history&lt;/code&gt;. There would likely be artefacts elsewhere on the system to give us a clue as to what they were trying to do, but that’s a different topic for another day.&lt;/p&gt;

&lt;h4 id=&quot;if-a-terminal-is-stopped-with-the-kill-command-commands-are-still-written-to-bash_history&quot;&gt;If a terminal is stopped with the &lt;code&gt;kill&lt;/code&gt; command, commands are still written to &lt;code&gt;.bash_history&lt;/code&gt;.&lt;/h4&gt;

&lt;p&gt;This one was simple enough to test. I ran the command &lt;code&gt;doesthiscommandappearinhistory?&lt;/code&gt; in a terminal, then opened a new terminal and killed it using its process ID, then checked &lt;code&gt;.bash_history&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;As you can see below, despite the Bash instance being killed, the command was still written to the log (the following &lt;code&gt;echo $$&lt;/code&gt; command was the one I used to identify the process ID of that terminal).&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/02/bash-history_2.png&quot; /&gt;&lt;/p&gt;

&lt;h4 id=&quot;however-if-the--sigkill-switch-is-used-commands-are-not-written-to-bash_history&quot;&gt;However, if the &lt;code&gt;-SIGKILL&lt;/code&gt; switch is used, commands are not written to &lt;code&gt;.bash_history&lt;/code&gt;.&lt;/h4&gt;

&lt;p&gt;More useful for red teamers - and something to bear in mind for blue teamers - is that &lt;code&gt;.bash_history&lt;/code&gt; is not written if the terminal is killed with the &lt;code&gt;-SIGKILL&lt;/code&gt; switch. Repeating the same experiment as above but with the additional switch meant no commands were written to disk.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/02/bash-history_3.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This is because by default &lt;code&gt;kill&lt;/code&gt; sends the &lt;code&gt;SIGTERM&lt;/code&gt; signal, which gracefully kills the process and allows Bash to write to &lt;code&gt;.bash_history&lt;/code&gt; as it is closing down. &lt;code&gt;SIGKILL&lt;/code&gt;, on the other hand, kills the process immediately before the commands can be written to the file.&lt;/p&gt;

&lt;h4 id=&quot;ssh-sessions-behave-in-a-similar-way-to-standard-terminals&quot;&gt;SSH sessions behave in a similar way to standard terminals.&lt;/h4&gt;

&lt;p&gt;If we run commands within an SSH session and then quit with &lt;code&gt;exit&lt;/code&gt; or by closing the window, &lt;code&gt;.bash_history&lt;/code&gt; is written in much the same way as it usually would be. The same also applies if we close the Command Prompt running the SSH session with Task Manager - presumably because Windows gives it time to tear down the connection in the background.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/02/bash-history_4.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Interestingly, if connecting via SSH from a Linux terminal, running a standard &lt;code&gt;kill&lt;/code&gt; command against the terminal will not close it until the user has ended the SSH session themselves. Using the &lt;code&gt;-SIGKILL&lt;/code&gt; switch ends the terminal process and SSH session immediately, but still writes the commands run during the session to the user’s &lt;code&gt;.bash_history&lt;/code&gt; file.&lt;/p&gt;

&lt;p&gt;This is likely because this is the work of &lt;code&gt;sshd&lt;/code&gt;, which sees its client kill the connection but itself closes gracefully. If we use &lt;code&gt;kill&lt;/code&gt; directly on &lt;code&gt;sshd&lt;/code&gt; itself - even with the &lt;code&gt;-SIGKILL&lt;/code&gt; switch - commands are still written to the file, which makes me wonder whether there is actually a way around this for SSH sessions.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/02/bash-history_5.png&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;finding-bash-commands-in-memory&quot;&gt;Finding Bash commands in memory&lt;/h3&gt;

&lt;p&gt;All the tests above showed when commands are and are not written to the on-disk &lt;code&gt;.bash_history&lt;/code&gt; file. Now let’s see what we can do when an attacker has an active session (and therefore commands in memory) but either &lt;code&gt;.bash_history&lt;/code&gt; has not been written yet or they have cleared its contents. This may also work for recently closed Bash terminal sessions that did not end gracefully.&lt;/p&gt;

&lt;p&gt;The first thing we need to do is take a memory image of the Linux system - for this we can use a tool called &lt;a href=&quot;https://github.com/504ensicsLabs/LiME&quot; target=&quot;_blank&quot;&gt;Linux Memory Extractor (LiME)&lt;/a&gt;. In a live incident scenario there are mechanisms to push the image to external storage or across the network, but I ran a basic command to create it locally.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;sudo insmod lime-4.18.0-240.22.1.el8_3.x86_64.ko “path=/mem-img.mem format=lime”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;To analyse our new memory image, we’ll use &lt;a href=&quot;https://www.volatilityfoundation.org/&quot; target=&quot;_blank&quot;&gt;Volatility&lt;/a&gt;, which is currently considered the pinnacle of memory forensics toolkits. It’s not quite as simple to run Volatility against Linux memory images as it is for Windows images. I won’t go into the full process here, but you need to &lt;a href=&quot;https://github.com/volatilityfoundation/volatility/wiki/Linux&quot; target=&quot;_blank&quot;&gt;create your own profile&lt;/a&gt; for the specific Linux distro and kernel version from which the image was captured.&lt;/p&gt;

&lt;p&gt;Once the profile is in place and ready to go, we can run Volatility’s &lt;code&gt;linux_bash&lt;/code&gt; module with the following command to search for Bash commands held in memory.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;python2 vol.py -f mem-img.mem –profile=LinuxCentOSx64 linux_bash&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;In the screenshot below, you can see a small extract from the results, are ordered first by the process ID of the Bash terminal to which the commands belonged and then by the time they were run. This includes commands that were run too recently to have been written to &lt;code&gt;.bash_history&lt;/code&gt;, and towards the bottom you’ll even see the commands I ran to set up and run LiME.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/02/bash-history_6.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;One small word of warning regarding timestamps… Everything from the &lt;code&gt;ls&lt;/code&gt; command at &lt;code&gt;15:59:18&lt;/code&gt; downwards appears to be correct, but you’ll probably notice that all the commands above that allegedly ran at exactly the same second, which is obviously not right. Further investigation is needed to work out why exactly that is, but it’s likely that there is some limit to the number of Bash command timestamps stored in memory, or that Volatility cannot always read them accurately.&lt;/p&gt;

&lt;h3 id=&quot;further-reading&quot;&gt;Further reading&lt;/h3&gt;

&lt;p&gt;The links below lead to pages that either inspired this post or provided useful information to compile it, including some more in-depth technical information on various features discussed above.&lt;/p&gt;

&lt;p&gt;• &lt;a href=&quot;https://www.gnu.org/software/bash/manual/html_node/Bash-History-Facilities.html&quot; target=&quot;_blank&quot;&gt;Bash history manual page&lt;/a&gt; (gnu.org)&lt;br /&gt;
• &lt;a href=&quot;https://www.geeksforgeeks.org/histcontrol-command-in-linux-with-examples/&quot; target=&quot;_blank&quot;&gt;$HISTCONTROL command in Linux with examples&lt;/a&gt; (geeksforgeeks.org)&lt;br /&gt;
• &lt;a href=&quot;https://linuxhandbook.com/sigterm-vs-sigkill/&quot; target=&quot;_blank&quot;&gt;SIGINT vs SIGKILL&lt;/a&gt; (linuxhandbook.com)&lt;br /&gt;
• &lt;a href=&quot;https://github.com/volatilityfoundation/volatility/wiki/Linux&quot; target=&quot;_blank&quot;&gt;Volatility Linux documentation&lt;/a&gt; (github.com)&lt;br /&gt;
• &lt;a href=&quot;https://www.crowdstrike.com/blog/how-to-extract-memory-information-to-spot-linux-malware/&quot; target=&quot;_blank&quot;&gt;How to extract memory information to spot Linux malware&lt;/a&gt; (crowdstrike.com)&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Updated 25/02/2022 to add section on the $HISTCONTROL variable.&lt;/em&gt;&lt;/p&gt;</content><author><name>mattcasmith</name></author><summary type="html">During any incident investigation on a Linux system, one of the most valuable things for responders and forensicators to establish is which commands were run. This is key to finding out what an attacker or malicious user was attempting to do, and what remediation activities are required. The .bash_history file, located in each user’s home directory, is usually the investigator’s first stop for this information. This file contains a list of Bash commands recently run by the user and may appear relatively simple at first glance, but there are many ins and outs to its behaviours. Most importantly, commands entered in a running Bash terminal are stored in memory and are only written to the file when the terminal is closed. As you can imagine, this means many caveats can be applied to the file’s contents, and in this post I aim to cover some of the most common scenarios, how they affect .bash_history, and alternatives when it does not contain the activity you are looking for. Contents 1. .bash_history behaviours           a. The $HISTCONTROL variable           b. The history command           c. Terminal windows and command order           d. Logical AND (&amp;amp;&amp;amp;)           e. Commands within scripts           f. The kill command           g. The -SIGKILL switch           h. SSH sessions 2. Finding Bash commands in memory 3. Further reading .bash_history behaviours To test exactly when commands are and aren’t recorded to the user’s .bash_history file, I ran a series of tests covering common scenarios in which commands might be run. All of these tests were run on a clean installation of CentOS. Here’s what I found… First, a note on the $HISTCONTROL variable… It is important to check the contents of the $HISTCONTROL variable on the subject system (after taking images and preserving evidence, so you’re not overwriting .bash_history with your own commands) as this has a potentially significant bearing on which commands will be written to the file. echo $HISTCONTROL The command above will return one of the following strings. The table below details which commands .bash_history will ignore when each of the possible values is present. $HISTCONTROL value Description ignorespace Excludes commands with a preceding space ignoredups Excludes subsequent duplicate commands ignorespace:ignoredups Excludes both commands with a preceding space and subsequent duplicates ignoreboth Excludes both commands with a preceding space and subsequent duplicates Therefore if the system has one of these values in the $HISTCONTROL variable, there may be commands missing from .bash_history. If it is set to ignorespace and the attacker is aware, they could even just slip in a space before each of their commands and write nothing to the history file at all! The CentOS system where I performed the testing for this post had $HISTCONTROL set to ignoredups. However, I also performed checks across other Linux distros I had available (namely Kali, Tsurugi, and Windows Subsystem for Linux) and it was set to ignoreboth for them all. So it’s important to check what you’re dealing with on the particular system you’re investigating. It’s also trivial to change the contents of the $HISTCONTROL variable to exclude more commands from .bash_history, so watch out for evidence it has changed, which will look something like the following: export HISTCONTROL=ignoreboth The history command outputs a combined history from .bash_history and memory. On a live Linux system, you might run the history command in a terminal window to review recently executed commands. The output of this command combines both the contents of .bash_history and any commands held in memory from the current session. In the example below, we can see that the test command doesthiscommandshowinhistory? is returned by the history command on the left, but has not yet been written to disk as shown on the right. Bear in mind that history will show commands from the memory of only the current Bash terminal - it won’t have access to those belonging to other processes. Since you won’t have access to the attacker’s terminal (unless the incident response team directly interrupted a hands-on-keyboard session), this means it won’t usually be much more useful than .bash_history itself. Commands are written to .bash_history in the order terminals are closed, not the order they are run. For this test, I opened two Bash terminals and ran example commands containing text stating which terminal they belonged to and in which order they were run. As I closed the first terminal before the second, you can see that both of its commands appear first in .bash_history, even though some of the second terminal’s commands were run first. It is the order the Bash processes end that matters. As was discussed before, note that if we had run the history command in one of these terminals, it would only have returned its own commands because each process has its own space in memory. Commands run with logical AND (&amp;amp;&amp;amp;) appear as a single entry in .bash_history. If we string several commands together using &amp;amp;&amp;amp; - a technique often used to save time (or in less innocent circumstances to avoid an attacker having to submit commands multiple times and risk attracting unwanted attention) - they will still only appear as one line in .bash_history. This is an important detail as investigators and forensicators, as it preserves the original context in which these commands were run. If they were run back-to-back in this manner, this invites the question: “Why?” Commands run within scripts are not reflected in .bash_history. To see whether commands in scripts are added to .bash_history or not, I created a script called folder_test.sh that would create a new folder, navigate into it, list the contents, navigate back up a level, and then delete the folder. You can see its contents in the left-hand window below. On the right, I ran the script with ./folder_test.sh. We know it executed successfully because the contents (i.e. nothing) are printed to the terminal. However, when we check the output of the history command we can see that although the command that ran the script was recorded, commands run within the script were not - an important distinction to remember when investigating Linux systems. This means that our attacker could potentially download or create a script named something innocuous like file_cleanup.sh, execute it, and we would be none the wiser as to what it did based on the contents of .bash_history. There would likely be artefacts elsewhere on the system to give us a clue as to what they were trying to do, but that’s a different topic for another day. If a terminal is stopped with the kill command, commands are still written to .bash_history. This one was simple enough to test. I ran the command doesthiscommandappearinhistory? in a terminal, then opened a new terminal and killed it using its process ID, then checked .bash_history. As you can see below, despite the Bash instance being killed, the command was still written to the log (the following echo $$ command was the one I used to identify the process ID of that terminal). However, if the -SIGKILL switch is used, commands are not written to .bash_history. More useful for red teamers - and something to bear in mind for blue teamers - is that .bash_history is not written if the terminal is killed with the -SIGKILL switch. Repeating the same experiment as above but with the additional switch meant no commands were written to disk. This is because by default kill sends the SIGTERM signal, which gracefully kills the process and allows Bash to write to .bash_history as it is closing down. SIGKILL, on the other hand, kills the process immediately before the commands can be written to the file. SSH sessions behave in a similar way to standard terminals. If we run commands within an SSH session and then quit with exit or by closing the window, .bash_history is written in much the same way as it usually would be. The same also applies if we close the Command Prompt running the SSH session with Task Manager - presumably because Windows gives it time to tear down the connection in the background. Interestingly, if connecting via SSH from a Linux terminal, running a standard kill command against the terminal will not close it until the user has ended the SSH session themselves. Using the -SIGKILL switch ends the terminal process and SSH session immediately, but still writes the commands run during the session to the user’s .bash_history file. This is likely because this is the work of sshd, which sees its client kill the connection but itself closes gracefully. If we use kill directly on sshd itself - even with the -SIGKILL switch - commands are still written to the file, which makes me wonder whether there is actually a way around this for SSH sessions. Finding Bash commands in memory All the tests above showed when commands are and are not written to the on-disk .bash_history file. Now let’s see what we can do when an attacker has an active session (and therefore commands in memory) but either .bash_history has not been written yet or they have cleared its contents. This may also work for recently closed Bash terminal sessions that did not end gracefully. The first thing we need to do is take a memory image of the Linux system - for this we can use a tool called Linux Memory Extractor (LiME). In a live incident scenario there are mechanisms to push the image to external storage or across the network, but I ran a basic command to create it locally. sudo insmod lime-4.18.0-240.22.1.el8_3.x86_64.ko “path=/mem-img.mem format=lime” To analyse our new memory image, we’ll use Volatility, which is currently considered the pinnacle of memory forensics toolkits. It’s not quite as simple to run Volatility against Linux memory images as it is for Windows images. I won’t go into the full process here, but you need to create your own profile for the specific Linux distro and kernel version from which the image was captured. Once the profile is in place and ready to go, we can run Volatility’s linux_bash module with the following command to search for Bash commands held in memory. python2 vol.py -f mem-img.mem –profile=LinuxCentOSx64 linux_bash In the screenshot below, you can see a small extract from the results, are ordered first by the process ID of the Bash terminal to which the commands belonged and then by the time they were run. This includes commands that were run too recently to have been written to .bash_history, and towards the bottom you’ll even see the commands I ran to set up and run LiME. One small word of warning regarding timestamps… Everything from the ls command at 15:59:18 downwards appears to be correct, but you’ll probably notice that all the commands above that allegedly ran at exactly the same second, which is obviously not right. Further investigation is needed to work out why exactly that is, but it’s likely that there is some limit to the number of Bash command timestamps stored in memory, or that Volatility cannot always read them accurately. Further reading The links below lead to pages that either inspired this post or provided useful information to compile it, including some more in-depth technical information on various features discussed above. • Bash history manual page (gnu.org) • $HISTCONTROL command in Linux with examples (geeksforgeeks.org) • SIGINT vs SIGKILL (linuxhandbook.com) • Volatility Linux documentation (github.com) • How to extract memory information to spot Linux malware (crowdstrike.com) Updated 25/02/2022 to add section on the $HISTCONTROL variable.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mattcasmith.net/wp-content/uploads/2022/02/bash-history_1.png" /><media:content medium="image" url="https://mattcasmith.net/wp-content/uploads/2022/02/bash-history_1.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">SANS Holiday Hack Challenge 2021: Slot machine walkthrough</title><link href="https://mattcasmith.net/2022/01/08/sans-holiday-hack-2021-slot-machine" rel="alternate" type="text/html" title="SANS Holiday Hack Challenge 2021&amp;#58; Slot machine walkthrough" /><published>2022-01-08T00:01:00+00:00</published><updated>2022-01-08T00:01:00+00:00</updated><id>https://mattcasmith.net/2022/01/08/sans-holiday-hack-2021-slot-machine</id><content type="html" xml:base="https://mattcasmith.net/2022/01/08/sans-holiday-hack-2021-slot-machine">&lt;p&gt;Here’s one more writeup from the &lt;a href=&quot;https://www.sans.org/mlp/holiday-hack-challenge/&quot; target=&quot;_blank&quot;&gt;SANS Holiday Hack Challenge&lt;/a&gt;! The slot machine hack was one of the showpiece challenges this year, so I thought I’d put together a quick blog post to guide you through the process of identifying and exploiting a vulnerability in the game.&lt;/p&gt;

&lt;h3 id=&quot;the-challenge&quot;&gt;The challenge&lt;/h3&gt;

&lt;p&gt;Our task is clear enough. We’re given a link to an online slot machine and the following request:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;Test the security of Jack Frost’s slot machines. What does the Jack Frost Tower casino security team threaten to do when your coin total exceeds 1,000? Submit the string in the server &lt;code&gt;data.response&lt;/code&gt; element.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Let’s follow the link and take a look at the slot machine. It’s fairly typical for such games (at least as per my understanding following &lt;a href=&quot;/2019/08/26/im-back-def-con-inspired-hacking/&quot;&gt;a short trip to Las Vegas for Black Hat and Def Con&lt;/a&gt;). You can choose various options before placing a bet by hitting the Spin button. Your credit is updated based on the outcome.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/01/sans-holiday-challenge-slots_1.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Since our objective is to hit more than 1,000 credit, we &lt;em&gt;could&lt;/em&gt; just play the slots for a long time to try to get lucky. But that would take forever. Instead, let’s take under the hood to see what we can hack…&lt;/p&gt;

&lt;h3 id=&quot;observing-requests&quot;&gt;Observing requests&lt;/h3&gt;

&lt;p&gt;Like any web application, the slot machine must be communicating with a server behind the scenes to make playing the game possible. By using the Burp Suite proxy’s intercept function, we can hold requests and responses between the web browser and the server to give us time to review their contents and see what’s going on when we place a bet. Here’s what is sent when we hit the Spin button…&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/01/sans-holiday-challenge-slots_2.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This looks like a pretty standard web request. We can see that this is a &lt;code&gt;POST&lt;/code&gt; request, which means that the client is transmitting information to the server (&lt;code&gt;slots.jackfrosttower.com&lt;/code&gt;). We can see that the request provides some cookie information, details about the page we’re on and our user agent, and so on. This is all pretty normal for this kind of request. But what about that last line?&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;n&quot;&gt;betamount&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;amp;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;numline&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;20&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;amp;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;cpl&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mf&quot;&gt;0.1&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;Comparing these variable names to the GUI, we can ascertain that this is our browser telling the server how much we’re betting, the number of lines we’re betting on, and the cost per line. Interesting. By tweaking these values, we might be able to:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Change the bet amount to a bigger stake than we can actually afford&lt;/li&gt;
  &lt;li&gt;Bet on more lines than actually exist on the slot machine&lt;/li&gt;
  &lt;li&gt;Adjust the cost per line to alter the slot machine’s financial calculations&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;From here, it’s a matter of trial and error. Attempting the first two potential exploits returns a &lt;code&gt;Server Error&lt;/code&gt; response. That leaves only one option - but how can we use the &lt;code&gt;cpl&lt;/code&gt; variable to our advantage?&lt;/p&gt;

&lt;h3 id=&quot;crafting-a-malicious-request&quot;&gt;Crafting a malicious request&lt;/h3&gt;

&lt;p&gt;So here’s the plan: We’re going to set &lt;code&gt;cpl&lt;/code&gt; to a negative value to confuse the game if this input isn’t validated. For example, if we set it to &lt;code&gt;-100&lt;/code&gt; and there are 20 lines to play, that spin would cost us -2,000 credits to play and therefore actually &lt;em&gt;increase&lt;/em&gt; our credit by 2,000 (and any winnings from the spin).&lt;/p&gt;

&lt;p&gt;To do this, we hit the Spin button on the slot machine’s GUI. Burp Suite intercepts the request. We must very quickly change the &lt;code&gt;cpl&lt;/code&gt; value to &lt;code&gt;-100&lt;/code&gt; and hit the Forward button before the spin times out.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/01/sans-holiday-challenge-slots_3.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;It looks like the server doesn’t check that &lt;code&gt;cpl&lt;/code&gt; is set to a valid value before executing the spin. All goes to plan and when the play is complete our credit is now in the thousands! Mission complete - almost.&lt;/p&gt;

&lt;p&gt;Our task was actually to find a value in the JSON response once our credit exceeds 1,000. To do this, we can take a look for the relevant request and response on Burp Suite’s HTTP history page. Sure enough, in the &lt;code&gt;data.response&lt;/code&gt; field, we can see the following message from casino security:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/wp-content/uploads/2022/01/sans-holiday-challenge-slots_4.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;And that’s our flag: &lt;code&gt;I'm going to have some bouncer trolls bounce you right out of this casino!&lt;/code&gt; Hopefully we have enough time to submit it and take the credit for our hacking first.&lt;/p&gt;

&lt;p&gt;If you found this walkthrough interesting, please consider &lt;a href=&quot;https://twitter.com/mattcasmith&quot; target=&quot;_blank&quot;&gt;following me on Twitter&lt;/a&gt;, and don’t forget to sign up for the next SANS Holiday Hack Challenge circa December 2022!&lt;/p&gt;</content><author><name>mattcasmith</name></author><summary type="html">Here’s one more writeup from the SANS Holiday Hack Challenge! The slot machine hack was one of the showpiece challenges this year, so I thought I’d put together a quick blog post to guide you through the process of identifying and exploiting a vulnerability in the game. The challenge Our task is clear enough. We’re given a link to an online slot machine and the following request: Test the security of Jack Frost’s slot machines. What does the Jack Frost Tower casino security team threaten to do when your coin total exceeds 1,000? Submit the string in the server data.response element. Let’s follow the link and take a look at the slot machine. It’s fairly typical for such games (at least as per my understanding following a short trip to Las Vegas for Black Hat and Def Con). You can choose various options before placing a bet by hitting the Spin button. Your credit is updated based on the outcome. Since our objective is to hit more than 1,000 credit, we could just play the slots for a long time to try to get lucky. But that would take forever. Instead, let’s take under the hood to see what we can hack… Observing requests Like any web application, the slot machine must be communicating with a server behind the scenes to make playing the game possible. By using the Burp Suite proxy’s intercept function, we can hold requests and responses between the web browser and the server to give us time to review their contents and see what’s going on when we place a bet. Here’s what is sent when we hit the Spin button… This looks like a pretty standard web request. We can see that this is a POST request, which means that the client is transmitting information to the server (slots.jackfrosttower.com). We can see that the request provides some cookie information, details about the page we’re on and our user agent, and so on. This is all pretty normal for this kind of request. But what about that last line? betamount=1&amp;amp;numline=20&amp;amp;cpl=0.1 Comparing these variable names to the GUI, we can ascertain that this is our browser telling the server how much we’re betting, the number of lines we’re betting on, and the cost per line. Interesting. By tweaking these values, we might be able to: Change the bet amount to a bigger stake than we can actually afford Bet on more lines than actually exist on the slot machine Adjust the cost per line to alter the slot machine’s financial calculations From here, it’s a matter of trial and error. Attempting the first two potential exploits returns a Server Error response. That leaves only one option - but how can we use the cpl variable to our advantage? Crafting a malicious request So here’s the plan: We’re going to set cpl to a negative value to confuse the game if this input isn’t validated. For example, if we set it to -100 and there are 20 lines to play, that spin would cost us -2,000 credits to play and therefore actually increase our credit by 2,000 (and any winnings from the spin). To do this, we hit the Spin button on the slot machine’s GUI. Burp Suite intercepts the request. We must very quickly change the cpl value to -100 and hit the Forward button before the spin times out. It looks like the server doesn’t check that cpl is set to a valid value before executing the spin. All goes to plan and when the play is complete our credit is now in the thousands! Mission complete - almost. Our task was actually to find a value in the JSON response once our credit exceeds 1,000. To do this, we can take a look for the relevant request and response on Burp Suite’s HTTP history page. Sure enough, in the data.response field, we can see the following message from casino security: And that’s our flag: I'm going to have some bouncer trolls bounce you right out of this casino! Hopefully we have enough time to submit it and take the credit for our hacking first. If you found this walkthrough interesting, please consider following me on Twitter, and don’t forget to sign up for the next SANS Holiday Hack Challenge circa December 2022!</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mattcasmith.net/wp-content/uploads/2022/01/sans-holiday-challenge-slots_1.png" /><media:content medium="image" url="https://mattcasmith.net/wp-content/uploads/2022/01/sans-holiday-challenge-slots_1.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry></feed>