Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[#64] Copy paste detection in lists #102

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

[#64] Copy paste detection in lists #102

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
317e3a6
Convert empty instances to `derive anyclass`
Heimdell Nov 9, 2021
e16377b
Move imports of DList in Xrefcheck.Core
Heimdell Nov 9, 2021
681c6b1
Add a type to store bad copy-paste info
Heimdell Nov 9, 2021
dfda710
Remove manual enumeration of flavours
Heimdell Nov 9, 2021
f60aece
Add a plug for copy-paste detector
Heimdell Nov 9, 2021
94748c5
Add copy-paste scanner and tests
Heimdell Nov 9, 2021
034bef5
Restore gist extraction
Heimdell Nov 9, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions src/Xrefcheck/Core.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,26 +102,36 @@ data Anchor = Anchor
deriving stock (Show, Eq, Generic)
deriving anyclass NFData

data CopyPaste = CopyPaste
{ cpAnchorText :: Text
Copy link
Contributor

@alyoanton9 alyoanton9 Nov 29, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like AnchorText and PlainText sound confusing. The issue doesn't focus on anchors, but all type of links, external and internal. Perhaps, more generic LinkText and URLText would be better?

UPD: Okay, I'm not sure if all the links were intended to check, as there is only example with files in issue description. But it's also not clear that this feature is for files only

, cpPlainText :: Text
, cpPosition :: Position
}
deriving stock (Show, Eq, Generic)
deriving anyclass NFData

data FileInfoDiff = FileInfoDiff
{ _fidReferences :: DList Reference
, _fidAnchors :: DList Anchor
, _fidCopyPastes :: DList CopyPaste
}
makeLenses ''FileInfoDiff

diffToFileInfo :: FileInfoDiff -> FileInfo
diffToFileInfo (FileInfoDiff refs anchors) =
FileInfo (DList.toList refs) (DList.toList anchors)
diffToFileInfo (FileInfoDiff refs anchors pastas) =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: I believe here should be pastes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍝

FileInfo (DList.toList refs) (DList.toList anchors) (DList.toList pastas)

instance Semigroup FileInfoDiff where
FileInfoDiff a b <> FileInfoDiff c d = FileInfoDiff (a <> c) (b <> d)
FileInfoDiff a b e <> FileInfoDiff c d f = FileInfoDiff (a <> c) (b <> d) (e <> f)

instance Monoid FileInfoDiff where
mempty = FileInfoDiff mempty mempty
mempty = FileInfoDiff mempty mempty mempty

-- | All information regarding a single file we care about.
data FileInfo = FileInfo
{ _fiReferences :: [Reference]
, _fiAnchors :: [Anchor]
, _fiCopyPastes :: [CopyPaste]
}
deriving stock (Show, Generic)
deriving anyclass NFData
Expand Down
16 changes: 9 additions & 7 deletions src/Xrefcheck/Scanners/Markdown.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,17 +144,19 @@ nodeExtractInfo input@(Node _ _ nSubs) = do
let aType = HeaderAnchor lvl
let aName = headerToAnchor flavor $ nodeExtractText node
let aPos = toPosition pos
return $ FileInfoDiff DList.empty $ DList.singleton $ Anchor {aType, aName, aPos}
return mempty
{ _fidAnchors = DList.singleton $ Anchor {aType, aName, aPos}
}

HTML_INLINE text -> do
let mName = T.stripSuffix "\">" =<< T.stripPrefix "<a name=\"" text
case mName of
Just aName -> do
let aType = HandAnchor
aPos = toPosition pos
return $ FileInfoDiff
mempty
(pure $ Anchor {aType, aName, aPos})
return mempty
{ _fidAnchors = DList.singleton $ Anchor {aType, aName, aPos}
}

Nothing -> do
return mempty
Expand All @@ -167,9 +169,9 @@ nodeExtractInfo input@(Node _ _ nSubs) = do
[t] -> (t, Nothing)
t : ts -> (t, Just $ T.intercalate "#" ts)
[] -> error "impossible"
return $ FileInfoDiff
(DList.singleton $ Reference {rName, rPos, rLink, rAnchor})
DList.empty
return mempty
{ _fidReferences = DList.singleton $ Reference {rName, rPos, rLink, rAnchor}
}

_ -> return mempty

Expand Down