cyber-mawonaj/web-core-fr-gp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: cyber-mawonaj/web-core-fr-gp:v1.3.0
Branches Tags
cyber-mawonaj/web-core-fr-gp:main cyber-mawonaj/web-core-fr-gp:version-2 cyber-mawonaj/web-core-fr-gp:develop
cyber-mawonaj/web-core-fr-gp:v1.8.0 cyber-mawonaj/web-core-fr-gp:v1.7.0 cyber-mawonaj/web-core-fr-gp:v1.6.0 cyber-mawonaj/web-core-fr-gp:v1.5.0 cyber-mawonaj/web-core-fr-gp:v1.4.1 cyber-mawonaj/web-core-fr-gp:v1.4.0 cyber-mawonaj/web-core-fr-gp:v1.3.0 cyber-mawonaj/web-core-fr-gp:v1.2.0 cyber-mawonaj/web-core-fr-gp:v1.1.0 cyber-mawonaj/web-core-fr-gp:v1.0.0
...
compare: cyber-mawonaj/web-core-fr-gp:v1.5.0
Branches Tags
cyber-mawonaj/web-core-fr-gp:main cyber-mawonaj/web-core-fr-gp:version-2 cyber-mawonaj/web-core-fr-gp:develop
cyber-mawonaj/web-core-fr-gp:v1.8.0 cyber-mawonaj/web-core-fr-gp:v1.7.0 cyber-mawonaj/web-core-fr-gp:v1.6.0 cyber-mawonaj/web-core-fr-gp:v1.5.0 cyber-mawonaj/web-core-fr-gp:v1.4.1 cyber-mawonaj/web-core-fr-gp:v1.4.0 cyber-mawonaj/web-core-fr-gp:v1.3.0 cyber-mawonaj/web-core-fr-gp:v1.2.0 cyber-mawonaj/web-core-fr-gp:v1.1.0 cyber-mawonaj/web-core-fr-gp:v1.0.0
41 Commits
v1.3.0 ... v1.5.0
Author SHA1 Message Date
Matt Baer 7dc8a92493 Catch all Bing and Googlebots, plus clean up 2023-07-06 18:38:34 -04:00
Matt Baer dee1e7cfbb Update bots list 
Also add and sort prefixes
 2023-07-06 18:29:39 -04:00
Matt Baer edb6b3fe63 Re-gofmt bots.go 2023-07-06 17:50:20 -04:00
Matt BaerandGitHub f06acefdb1 Merge pull request #16 from writeas/dependabot/go_modules/github.com/microcosm-cc/bluemonday-1.0.23 
Bump github.com/microcosm-cc/bluemonday from 1.0.5 to 1.0.23
 2023-04-06 08:06:55 -04:00
dependabot[bot]andGitHub 33b431fb41 Bump github.com/microcosm-cc/bluemonday from 1.0.5 to 1.0.23 
Bumps [github.com/microcosm-cc/bluemonday](https://github.com/microcosm-cc/bluemonday) from 1.0.5 to 1.0.23.
- [Release notes](https://github.com/microcosm-cc/bluemonday/releases)
- [Commits](https://github.com/microcosm-cc/bluemonday/compare/v1.0.5...v1.0.23)

---
updated-dependencies:
- dependency-name: github.com/microcosm-cc/bluemonday
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-04-04 16:11:47 +00:00
Matt Baer 4934315624 Reformat example code on PostDescription 2023-04-04 12:10:40 -04:00
Matt Baer 4e8d67e937 Update go.sum with go-strip-markdown 2023-04-04 12:10:20 -04:00
Matt Baer 9e2b7620b1 Add funcs PostTitle, GetSlug, GetSlugFromPost 2023-04-04 12:09:49 -04:00
Matt BaerandGitHub b5516d22e1 Merge pull request #15 from writeas/fix-key-decoding 
Support decoding PKCS#8 keys
 2022-11-11 01:58:35 -05:00
Matt Baer 73b029dffa Support decoding PKCS#8 keys 
These are generated by OpenSSL v3. Previously, key decoding would fail,
breaking federation.

This fixes #14
 2022-11-11 01:56:53 -05:00
Matt Baer ab8ed5dd58 Support sending out activities around Persons 2022-05-11 18:06:23 -04:00
Matt Baer 0da0bcaf01 Support HTML links with rel attr in ApplyBasicAccessibleMarkdown 
Ref T874 T744
 2022-01-18 16:27:28 -05:00
Matt Baer 3e1dcd609c Restore default title dash 2022-01-06 13:35:11 -05:00
Matt Baer 251490523d Add ApplyBasicAccessibleMarkdown func 2021-12-15 15:59:22 -05:00
Matt Baer 789795d7fb Update Japanese translations 2021-11-05 11:00:54 -04:00
Matt Baer 482d2a0340 Update Italian translations 2021-11-05 10:59:12 -04:00
Matt Baer b0d08b0b2d Add Hebrew translations 2021-11-05 10:57:53 -04:00
Matt Baer 4ea5f505e1 Update Greek translations 2021-11-05 10:56:23 -04:00
Matt Baer 6594abfd76 Update German translations 2021-11-05 10:55:04 -04:00
Matt Baer 8e58a42997 Add Galician translations 2021-11-05 10:52:59 -04:00
Matt Baer ed63d3d637 Add Basque translations 2021-11-05 10:51:06 -04:00
Matt Baer 61c88f6532 Update Esperanto translation 2021-11-05 10:49:53 -04:00
Matt Baer 05ae34b399 Add Dutch translations 2021-11-05 10:47:36 -04:00
Matt Baer 11821b1fe1 Add Danish translations 2021-11-05 10:45:30 -04:00
Matt Baer 26a278eba8 Add new phrases (defaults) 2021-11-05 10:43:35 -04:00
Matt Baer 3b7d4deeaa Add Czech translations 2021-11-05 10:41:30 -04:00
Matt Baer 44e795b77d Update Chinese translation 2021-11-05 10:38:14 -04:00
Matt Baer 982e31c5be Update Arabic translation 2021-11-05 10:32:33 -04:00
Matt Baer c643a95bda Update French translations 2021-11-05 10:30:08 -04:00
Matt Baer ba85819607 Add PostCount property to Category 2021-11-03 13:59:21 -04:00
Matt Baer f276f4d64a Add NewCategoryFromPartial func 2021-10-16 15:27:37 -04:00
Matt Baer 375d22d230 Add HashtagFromTitle helper 2021-10-16 13:19:00 -04:00
Matt Baer bec10e41b5 Merge branch 'main' of github.com:writeas/web-core into main 2021-10-15 15:38:45 -04:00
Matt Baer a8daef8401 Add category pkg 
This supports categories in Write.as / WriteFreely.

Ref T809
 2021-09-29 17:01:17 -04:00
Matt BaerandGitHub ee6bf8e8b7 Merge pull request #12 from writeas/dependabot/go_modules/github.com/microcosm-cc/bluemonday-1.0.5 
Bump github.com/microcosm-cc/bluemonday from 1.0.2 to 1.0.5
 2021-09-13 14:39:52 -05:00
dependabot[bot]andGitHub bb86406b9d Bump github.com/microcosm-cc/bluemonday from 1.0.2 to 1.0.5 
Bumps [github.com/microcosm-cc/bluemonday](https://github.com/microcosm-cc/bluemonday) from 1.0.2 to 1.0.5.
- [Release notes](https://github.com/microcosm-cc/bluemonday/releases)
- [Commits](https://github.com/microcosm-cc/bluemonday/compare/v1.0.2...v1.0.5)

---
updated-dependencies:
- dependency-name: github.com/microcosm-cc/bluemonday
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-09-13 19:34:04 +00:00
Matt Baer 28cf6b3866 Fix TestNewImageAttachment tests 2021-09-13 15:32:27 -04:00
Matt Baer 8b45bd4fcc Support creating Document attachments 
In particular, this will support audio attachments.

Ref T872
 2021-09-13 15:32:10 -04:00
Matt Baer 5fb147fe78 Add PostLede and TruncToWord tests from WriteFreely 2021-04-21 11:50:00 -04:00
Matt Baer fd1559928a Copy post parsing funcs to posts pkg 2021-04-13 23:49:59 -04:00
Matt Baer 95a3a717ed Move remaining writeas/nerds/store funcs to web-core 
Finishes the work started in #8.
 2021-03-30 12:44:22 -04:00
31 changed files with 1246 additions and 293 deletions
Expand all files Collapse all files
activitypub/keys.go
+2 -2
View File
@@ -69,8 +69,8 @@ func parsePublicKey(der []byte) (crypto.PublicKey, error) {
// them in that order.
func DecodePrivateKey(k []byte) (crypto.PrivateKey, error) {
block, _ := pem.Decode(k)
if block == nil || block.Type != "RSA PRIVATE KEY" {
return nil, fmt.Errorf("failed to decode PEM block containing private key")
if block == nil || (block.Type != "RSA PRIVATE KEY" && block.Type != "PRIVATE KEY") {
return nil, fmt.Errorf("failed to decode PEM block containing private key, type %s", block.Type)
}
return parsePrivateKey(block.Bytes)
activitystreams/activity.go
+26 -6
View File
@@ -106,19 +106,29 @@ func NewFollowActivity(actorIRI, followeeIRI string) *FollowActivity {
// Object is the primary base type for the Activity Streams vocabulary.
type Object struct {
BaseObject
Published time.Time `json:"published"`
Published time.Time `json:"published,omitempty"`
Summary *string `json:"summary,omitempty"`
InReplyTo *string `json:"inReplyTo"`
InReplyTo *string `json:"inReplyTo,omitempty"`
URL string `json:"url"`
AttributedTo string `json:"attributedTo"`
To []string `json:"to"`
AttributedTo string `json:"attributedTo,omitempty"`
To []string `json:"to,omitempty"`
CC []string `json:"cc,omitempty"`
Name string `json:"name,omitempty"`
Content string `json:"content"`
Content string `json:"content,omitempty"`
ContentMap map[string]string `json:"contentMap,omitempty"`
Tag []Tag `json:"tag"`
Tag []Tag `json:"tag,omitempty"`
Attachment []Attachment `json:"attachment,omitempty"`
// Person
Inbox string `json:"inbox,omitempty"`
Outbox string `json:"outbox,omitempty"`
Following string `json:"following,omitempty"`
Followers string `json:"followers,omitempty"`
PreferredUsername string `json:"preferredUsername,omitempty"`
Icon *Image `json:"icon,omitempty"`
PublicKey *PublicKey `json:"publicKey,omitempty"`
Endpoints *Endpoints `json:"endpoints,omitempty"`
// Extensions
// NOTE: add extensions here
}
@@ -150,3 +160,13 @@ func NewArticleObject() *Object {
}
return &o
}
// NewPersonObject creates a basic Person object.
func NewPersonObject() *Object {
o := Object{
BaseObject: BaseObject{
Type: "Person",
},
}
return &o
}
activitystreams/attachment.go
+19 -5
View File
@@ -14,20 +14,34 @@ type Attachment struct {
type AttachmentType string
const AttachImage AttachmentType = "Image"
const (
AttachImage AttachmentType = "Image"
AttachDocument AttachmentType = "Document"
)
// NewImageAttachment creates a new Attachment from the given URL, setting the
// correct type and automatically detecting the MediaType based on the file
// extension.
func NewImageAttachment(url string) Attachment {
var imgType string
return newAttachment(url, AttachImage)
}
// NewDocumentAttachment creates a new Attachment from the given URL, setting the
// correct type and automatically detecting the MediaType based on the file
// extension.
func NewDocumentAttachment(url string) Attachment {
return newAttachment(url, AttachDocument)
}
func newAttachment(url string, attachType AttachmentType) Attachment {
var fileType string
extIdx := strings.LastIndexByte(url, '.')
if extIdx > -1 {
imgType = mime.TypeByExtension(url[extIdx:])
fileType = mime.TypeByExtension(url[extIdx:])
}
return Attachment{
Type: AttachImage,
Type: attachType,
URL: url,
MediaType: imgType,
MediaType: fileType,
}
}
activitystreams/attachment_test.go
+28 -4
View File
@@ -12,19 +12,19 @@ func TestNewImageAttachment(t *testing.T) {
tests := []struct {
name string
args args
want *Attachment
want Attachment
}{
{name: "good svg", args: args{"https://writefreely.org/img/writefreely.svg"}, want: &Attachment{
{name: "good svg", args: args{"https://writefreely.org/img/writefreely.svg"}, want: Attachment{
Type: "Image",
URL: "https://writefreely.org/img/writefreely.svg",
MediaType: "image/svg+xml",
}},
{name: "good png", args: args{"https://i.snap.as/12345678.png"}, want: &Attachment{
{name: "good png", args: args{"https://i.snap.as/12345678.png"}, want: Attachment{
Type: "Image",
URL: "https://i.snap.as/12345678.png",
MediaType: "image/png",
}},
{name: "no extension", args: args{"https://i.snap.as/12345678"}, want: &Attachment{
{name: "no extension", args: args{"https://i.snap.as/12345678"}, want: Attachment{
Type: "Image",
URL: "https://i.snap.as/12345678",
MediaType: "",
@@ -38,3 +38,27 @@ func TestNewImageAttachment(t *testing.T) {
})
}
}
func TestNewDocumentAttachment(t *testing.T) {
type args struct {
url string
}
tests := []struct {
name string
args args
want Attachment
}{
{name: "mp3", args: args{"https://listen.as/matt/abc.mp3"}, want: Attachment{
Type: "Document",
URL: "https://listen.as/matt/abc.mp3",
MediaType: "audio/mpeg",
}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := NewDocumentAttachment(tt.args.url); !reflect.DeepEqual(got, tt.want) {
t.Errorf("NewDocumentAttachment() = %+v, want %+v", got, tt.want)
}
})
}
}
bots/bots.go
+357 -150
View File
@@ -7,140 +7,272 @@ import "strings"
var bots = map[string]bool{
"ABACHOBot/8.14 (Windows NT 6.1 1.5; ko;)": true,
"AcademicBotRTU (https://academicbot.rtu.lv; mailto:caps@rtu.lv)": true,
"AddThis.com robot tech.support@clearspring.com": true,
"AdsBot-Google (+http://www.google.com/adsbot.html)": true,
"AdsTxtCrawlerTP/1.2": true,
"Alwyzbot/1.0": true,
"Apache-HttpClient/4.2.2 (java 1.5)": true,
"Amazon-Advertising-ad-standards-bot/1.0": true,
"Apple Color Emoji": true,
"AwarioRssBot/1.0 (+https://awario.com/bots.html; bots@awario.com)": true,
"AwarioSmartBot/1.0 (+https://awario.com/bots.html; bots@awario.com)": true,
"BeeperBot/0 Matrix-Media-Repo/1": true,
"bidswitchbot/1.0": true,
"bitlybot": true,
"bitlybot/3.0 (+http://bit.ly/)": true,
"BrightEdge Crawler/1.0 (crawler@brightedge.com)": true,
"BSbot 1.1 (monthly copyright check - html/js/css)": true,
"Buzzbot/1.0 (Buzzbot; http://www.buzzstream.com; buzzbot@buzzstream.com)": true,
"cis455crawler": true,
"COMODO SSL Checker": true,
"crawlernutchtest/Nutch-1.9": true,
"CRAZYWEBCRAWLER 0.9.8, http://www.crazywebcrawler.com": true,
"CheckMarkNetwork/1.0 (+http://www.checkmarknetwork.com/spider.html)": true,
"Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)": true,
"cis455crawler": true,
"Clickagy Intelligence Bot v2": true,
"COMODO SSL Checker": true,
"companyspotter/1.0.0.0 (robot@companyspotter.com)": true,
"crawler_eb_germany_2.0": true,
"crawlernutchtest/Nutch-1.9": true,
"CRAZYWEBCRAWLER 0.9.8, http://www.crazywebcrawler.com": true,
"CSS Certificate Spider (http://www.css-security.com/certificatespider/)": true,
"datebot": true,
"DeadYetBot/1.0 (+http://deadyet.lol)": true,
"dj-research/Nutch-1.11 (analytics@@demandjump.com)": true,
"datebot": true,
"DeadYetBot/1.0 (+http://deadyet.lol)": true,
"Deskyobot/1.0 (+https://www.deskyo.com/bot)": true,
"DingTalkBot-LinkService/1.0 (+https://open-doc.dingtalk.com/microapp/faquestions/ftpfeu)": true,
"DisqusAdstxtCrawler/0.1 (+https://help.disqus.com/en/articles/1765357-ads-txt-implementation-guide)": true,
"dj-research/Nutch-1.11 (analytics@@demandjump.com)": true,
"DnBCrawler-Analytics": true,
"DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)": true,
"Domain Re-Animator Bot (http://domainreanimator.com) - support@domainreanimator.com": true,
"DomainCrawler/1.0": true,
"Domain Re-Animator Bot (http://domainreanimator.com) - support@domainreanimator.com": true,
"ExactSeekCrawler/1.0": true,
"Gigabot/1.0": true,
"Go 1.1 package http": true,
"Googlebot/2.1 (+http://www.googlebot.com/bot.html)": true,
"Googlebot/2.1 (+http://www.google.com/bot.html)": true,
"Googlebot-Image/1.0": true,
"libwww-perl/6.04": true,
"LinkedInBot/1.0 (compatible; Mozilla/5.0; Jakarta Commons-HttpClient/3.1 +http://www.linkedin.com)": true,
"LivelapBot/0.2 (http://site.livelap.com/crawler)": true,
"LSSRocketCrawler/1.0 LightspeedSystems": true,
"Mediatoolkitbot (complaints@mediatoolkit.com)": true,
"Mediatoolkitbot (info@mediatoolkit.com)": true,
"Mediumbot-MetaTagFetcher/0.1 (+https://medium.com/)": true,
"Melvil/1.0": true,
"MetaCommentBot; http://metacomment.io/about": true,
"mfibot/1.1 (http://www.mfisoft.ru/analyst/; <admin@mfisoft.ru>; en-RU)": true,
"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview Analytics) Chrome/27.0.1453 Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 AppEngine-Google; (+http://code.google.com/appengine; appid: s~feedly-social)": true,
"Mozilla/5.0/Chrome/12.0.742.112 (Macintosh; Intel Mac OS X 10_6_8)": true,
"Mozilla/5.0 (compatible; aiHitBot/2.9; +https://www.aihitdata.com/about)": true,
"Mozilla/5.0 (compatible; alexa site audit/1.0; +http://www.alexa.com/help/webmasters; )": true,
"Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)": true,
"Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)": true,
"Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)": true,
"Mozilla/5.0 (compatible; BuzzSumo; +http://www.buzzsumo.com/bot.html)": true,
"Mozilla/5.0 (compatible; coccoc/1.0; +http://help.coccoc.com/)": true,
"Mozilla/5.0 (compatible; Discordbot/1.0; +https://discordapp.com)": true,
"Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)": true,
"Mozilla/5.0 (compatible; DuckDuckGo-Favicons-Bot/1.0; +http://duckduckgo.com)": true,
"Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)": true,
"Mozilla/5.0 (compatible; Feedspotbot/1.0; +http://www.feedspot.com/fs/bot)": true,
"Mozilla/5.0 (compatible; Findxbot/1.0; +http://www.findxbot.com)": true,
"Mozilla/5.0 (compatible; Gluten Free Crawler/1.0; +http://glutenfreepleasure.com/)": true,
"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)": true,
"Mozilla/5.0 (compatible; heritrix/3.3.0-SNAPSHOT-20140702-2247 +http://archive.org/details/archive.org_bot)": true,
"Mozilla/5.0 (compatible; heritrix/3.3.0-SNAPSHOT-2014-11-14T15:29:34Z +http://citeseerx.ist.psu.edu/)": true,
"Mozilla/5.0/(compatible; heritrix/3.3.0-SNAPSHOT-20150803-2130 +http://literatur-im-netz.dla-marbach.de)": true,
"Mozilla/5.0 (compatible; Kraken/0.1; http://linkfluence.net/; bot@linkfluence.net)": true,
"Mozilla/5.0 (compatible; linkdexbot/2.2; +http://www.linkdex.com/bots/)": true,
"Mozilla/5.0 (compatible; LinkFeatureBot)": true,
"Mozilla/5.0 (compatible; LinkisBot/1.0; bot@linkis.com)": true,
"Mozilla/5.0 (compatible; LinkisBot/1.0; bot@linkis.com) (iPhone; CPU iPhone OS 8_4_1 like Mac OS X) Mobile/12H321": true,
"Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)": true,
"Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Robots/2.0; +http://go.mail.ru/help/robots)": true,
"Mozilla/5.0 (compatible; Lipperhey-Kaus-Australis/5.0; +https://www.lipperhey.com/en/about/)": true,
"Mozilla/5.0 (compatible; meanpathbot/1.0; +http://www.meanpath.com/meanpathbot.html)": true,
"Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)": true,
"Mozilla/5.0 (compatible; MixrankBot; crawler@mixrank.com)": true,
"Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)": true,
"Mozilla/5.0 (compatible; MojeekBot/0.6; +https://www.mojeek.com/bot.html)": true,
"Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)": true,
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; abot v1.2.3.1 http://code.google.com/p/abot)": true,
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider": true,
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)": true,
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)": true,
"Mozilla/5.0 (compatible; MSIE or Firefox mutant;) Daum 4.1": true,
"Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server;) Daum 4.1": true,
"Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server;) Daumoa 4.0": true,
"Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)": true,
"Mozilla/5.0 (compatible; oBot/2.3.1; +http://filterdb.iss.net/crawler/)": true,
"Mozilla/5.0 (compatible; OpenHoseBot/2.1; +http://www.openhose.org/bot.html)": true,
"Mozilla/5.0 (compatible; OpsBot/1.0)": true,
"Mozilla/5.0 (compatible; OptimizationCrawler/0.2; +http://www.domainoptima.com/robot)": true,
"Mozilla/5.0 (compatible; PaperLiBot/2.1; http://support.paper.li/entries/20023257-what-is-paper-li)": true,
"Mozilla/5.0 (compatible; PrivacyAwareBot/1.1; +http://www.privacyaware.org)": true,
"Mozilla/5.0 (compatible; Qwantify/2.2w; +https://www.qwant.com/)/*": true,
"Mozilla/5.0 (compatible; redditbot/1.0; +http://www.reddit.com/feedback)": true,
"Mozilla/5.0 (compatible; Sadakura; +http://shorf.com/bot.php)": true,
"Mozilla/5.0 (compatible; SemrushBot/1.1~bl; +http://www.semrush.com/bot.html)": true,
"Mozilla/5.0 (compatible; SemrushBot/1~bl; +http://www.semrush.com/bot.html)": true,
"Mozilla/5.0 (compatible; SEOkicks-Robot; +http://www.seokicks.de/robot.html)": true,
"Mozilla/5.0 (compatible; SEOlyticsCrawler/3.0; +http://crawler.seolytics.net/)": true,
"Mozilla/5.0 (compatible; SeznamBot/3.2; +http://fulltext.sblog.cz/)": true,
"Mozilla/5.0 (compatible; SeznamBot/3.2; +http://napoveda.seznam.cz/en/seznambot-intro/)": true,
"Mozilla/5.0 (compatible; spbot/4.4.2; +http://OpenLinkProfiler.org/bot )": true,
"Mozilla/5.0 (compatible; spbot/5.0.1; +http://OpenLinkProfiler.org/bot )": true,
"Mozilla/5.0 (compatible; spbot/5.0.2; +http://OpenLinkProfiler.org/bot )": true,
"Mozilla/5.0 (compatible; spbot/5.0; +http://OpenLinkProfiler.org/bot )": true,
"Mozilla/5.0 (compatible; special_archiver/3.1.1 +http://www.archive.org/details/archive.org_bot)": true,
"Mozilla/5.0 (compatible; TestCrawler)": true,
"Mozilla/5.0 (compatible; uMBot-LN/1.0; mailto: crawling@ubermetrics-technologies.com)": true,
"Mozilla/5.0 (compatible; UptimeRobot/2.0; http://www.uptimerobot.com/)": true,
"Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/)": true,
"Mozilla/5.0 (compatible; woriobot +http://worio.com)": true,
"Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)": true,
"Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)": true,
"Mozilla/5.0 (compatible; Yeti/1.1; +http://help.naver.com/robots/)": true,
"Mozilla/5.0 (compatible; zitebot support [at] zite [dot] com +http://zite.com)": true,
"Mozilla/5.0/Firefox/42.0 (contactbigdatafr at gmail.com)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Applebot/0.3; +http://www.apple.com/go/applebot)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; http://www.bing.com/bingbot.htm)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4 (compatible; Laserlikebot/0.1)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13F69 Safari/601.1": true,
"Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)": true,
"Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:42.0) Gecko/20100101 Firefox/42.0": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0 (FlipboardProxy/1.1; +http://flipboard.com/browserproxy)": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:29.0) Gecko/20100101 Firefox/29.0": true,
"Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5": true,
"DomainStatsBot/1.0 (https://domainstats.com/pages/our-bot)": true,
"drinespider/Nutch-1.19 (D-RINE Spider; www.d-rine.com/search/about; www.d-rine.com/contact)": true,
"DuckDuckBot-Https/1.1; (+https://duckduckgo.com/duckduckbot)": true,
"ExactSeekCrawler/1.0": true,
"EyeMonIT Uptime Bot Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36": true,
"FediCrawl/1.0": true,
"fedistatsCrawler/1.0": true,
"FeedlyBot/1.0 (http://feedly.com)": true,
"GG PeekBot 2.0 ( https://www.gg.pl/ https://www.gg.pl/info/praca/ )": true,
"Gigabot/1.0": true,
"GNUsocialBot 2.0.1-beta0 - https://gnusocial.rocks": true,
"Go 1.1 package http": true,
"Go-http-client/2.0": true,
"Google-Adwords-Instant (+http://www.google.com/adsbot.html)": true,
"Google-Display-Ads-Bot": true,
"Googlebot": true,
"Greppr Bot 1.0/Nutch-1.19": true,
"GroupMeBot/1.0": true,
"GuzzleHttp/7": true,
"Gwene/1.0 (The gwene.org rss-to-news gateway) Googlebot": true,
"HubSpot Connect 2.0 (http://dev.hubspot.com/) (namespace: hs_web_crawler) - RoadsModelsJobs-hubspot-53-domain-web-crawl": true,
"ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com)": true,
"IABot/2.0 (+https://meta.wikimedia.org/wiki/InternetArchiveBot/FAQ_for_sysadmins) (Checking if link from Wikipedia is broken and needs removal)": true,
"IonCrawl (https://www.ionos.de/terms-gtc/faq-crawler-en/)": true,
"LightspeedSystemsCrawler Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US)": true,
"LinkedInBot/1.0 (compatible; Mozilla/5.0; Apache-HttpClient +http://www.linkedin.com)": true,
"LinkedInBot/1.0 (compatible; Mozilla/5.0; Jakarta Commons-HttpClient/3.1 +http://www.linkedin.com)": true,
"LivelapBot/0.2 (http://site.livelap.com/crawler)": true,
"LSSRocketCrawler/1.0 LightspeedSystems": true,
"magpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net)": true,
"Mastodon server indexer": true,
"MBCrawler/1.0 (https://monitorbacklinks.com/robot)": true,
"Mediatoolkitbot (complaints@mediatoolkit.com)": true,
"Mediatoolkitbot (info@mediatoolkit.com)": true,
"Melvil/1.0": true,
"MetaCommentBot; http://metacomment.io/about": true,
"mfibot/1.1 (http://www.mfisoft.ru/analyst/; <admin@mfisoft.ru>; en-RU)": true,
"Minoru's Fediverse Crawler (+https://nodes.fediverse.party)": true,
"mj12bot": true,
"Mozilla/4.0 (compatible; QwertyNetworksBot/1.0; +https://qwertynetworks.com/faq/QwertyNetworksBot)": true,
"Mozilla/4.0 (compatible; Wibybot; https://wiby.me/)": true,
"Mozilla/5.0 (Android 11; Mobile; rv:109.0) Gecko/114.0 Firefox/114.0": true,
"Mozilla/5.0 (compatible) SemanticScholarBot (+https://www.semanticscholar.org/crawler)": true,
"Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)": true,
"Mozilla/5.0 (compatible; aiHitBot/2.9; +https://www.aihitdata.com/about)": true,
"Mozilla/5.0 (compatible; alexa site audit/1.0; +http://www.alexa.com/help/webmasters; )": true,
"Mozilla/5.0 (compatible; AmazonAdBot/1.0; +https://adbot.amazon.com)": true,
"Mozilla/5.0 (compatible; archive.org_bot +http://archive.org/details/archive.org_bot) Zeno/0030699 warc/v0.8.32": true,
"Mozilla/5.0 (compatible; Atomseobot/2.0; +http://https://error404.atomseo.com/)": true,
"Mozilla/5.0 (compatible; AwarioBot/1.0; +https://awario.com/bots.html)": true,
"Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html": true,
"Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)": true,
"Mozilla/5.0 (compatible; BitSightBot/1.0)": true,
"Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)": true,
"Mozilla/5.0 (compatible; BomboraBot/1.0; +http://www.bombora.com/bot)": true,
"Mozilla/5.0 (compatible; BuzzSumo; +http://www.buzzsumo.com/bot.html)": true,
"Mozilla/5.0 (compatible; Bytespider; spider-feedback@bytedance.com) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.0.0 Safari/537.36": true,
"Mozilla/5.0 (compatible; coccoc/1.0; +http://help.coccoc.com/)": true,
"Mozilla/5.0 (compatible; coccocbot-image/1.0; +http://help.coccoc.com/searchengine)": true,
"Mozilla/5.0 (compatible; coccocbot-web/1.0; +http://help.coccoc.com/searchengine)": true,
"Mozilla/5.0 (compatible; Cocolyzebot/1.0; https://cocolyze.com/bot)": true,
"Mozilla/5.0 (compatible; DataForSeoBot/1.0; +https://dataforseo.com/dataforseo-bot)": true,
"Mozilla/5.0 (compatible; Discordbot/1.0; +https://discordapp.com)": true,
"Mozilla/5.0 (compatible; Discordbot/2.0; +https://discordapp.com)": true,
"Mozilla/5.0 (compatible; Dmbot/1.1)": true,
"Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)": true,
"Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)": true,
"Mozilla/5.0 (compatible; DuckDuckGo-Favicons-Bot/1.0; +http://duckduckgo.com)": true,
"Mozilla/5.0 (compatible; ev-crawler/1.0; +https://headline.com/legal/crawler)": true,
"Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)": true,
"Mozilla/5.0 (compatible; Feedspotbot/1.0; +http://www.feedspot.com/fs/bot)": true,
"Mozilla/5.0 (compatible; FFZBot/2.4.1; +https://www.frankerfacez.com)": true,
"Mozilla/5.0 (compatible; FFZBot/3.0.0; +https://www.frankerfacez.com)": true,
"Mozilla/5.0 (compatible; Findxbot/1.0; +http://www.findxbot.com)": true,
"Mozilla/5.0 (compatible; Gluten Free Crawler/1.0; +http://glutenfreepleasure.com/)": true,
"Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php)": true,
"Mozilla/5.0 (compatible; heritrix/3.3.0-SNAPSHOT-2014-11-14T15:29:34Z +http://citeseerx.ist.psu.edu/)": true,
"Mozilla/5.0 (compatible; heritrix/3.3.0-SNAPSHOT-20140702-2247 +http://archive.org/details/archive.org_bot)": true,
"Mozilla/5.0 (compatible; Hkfl-Bot/%s +https://hackerfall.com/)": true,
"Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Exabot-Thumbnails)": true,
"Mozilla/5.0 (compatible; Kraken/0.1; http://linkfluence.net/; bot@linkfluence.net)": true,
"Mozilla/5.0 (compatible; Linespider/1.1; +https://lin.ee/4dwXkTH)": true,
"Mozilla/5.0 (compatible; linkdexbot/2.0; +http://www.linkdex.com/bots/)": true,
"Mozilla/5.0 (compatible; linkdexbot/2.2; +http://www.linkdex.com/bots/)": true,
"Mozilla/5.0 (compatible; LinkFeatureBot)": true,
"Mozilla/5.0 (compatible; LinkisBot/1.0; bot@linkis.com) (iPhone; CPU iPhone OS 8_4_1 like Mac OS X) Mobile/12H321": true,
"Mozilla/5.0 (compatible; LinkisBot/1.0; bot@linkis.com)": true,
"Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)": true,
"Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Fast/2.0; +https://help.mail.ru/webmaster/indexing/robots)": true,
"Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Img/2.0; +https://help.mail.ru/webmaster/indexing/robots)": true,
"Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Robots/2.0; +http://go.mail.ru/help/robots)": true,
"Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Robots/2.0; +https://help.mail.ru/webmaster/indexing/robots)": true,
"Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Target/2.0; +https://help.mail.ru/webmaster/indexing/robots)": true,
"Mozilla/5.0 (compatible; Lipperhey-Kaus-Australis/5.0; +https://www.lipperhey.com/en/about/)": true,
"Mozilla/5.0 (compatible; meanpathbot/1.0; +http://www.meanpath.com/meanpathbot.html)": true,
"Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)": true,
"Mozilla/5.0 (compatible; MixrankBot; crawler@mixrank.com)": true,
"Mozilla/5.0 (compatible; MJ12bot/v1.4.2; http://www.majestic12.co.uk/bot.php?+)": true,
"Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)": true,
"Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)": true,
"Mozilla/5.0 (compatible; MojeekBot/0.11; +https://www.mojeek.com/bot.html)": true,
"Mozilla/5.0 (compatible; MojeekBot/0.6; +https://www.mojeek.com/bot.html)": true,
"Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)": true,
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; abot v1.2.3.1 http://code.google.com/p/abot)": true,
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider": true,
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)": true,
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)": true,
"Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server;) Daum 4.1": true,
"Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server;) Daumoa 4.0": true,
"Mozilla/5.0 (compatible; MSIE or Firefox mutant;) Daum 4.1": true,
"Mozilla/5.0 (compatible; NetpeakCheckerBot/3.6; +https://netpeaksoftware.com/checker)": true,
"Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)": true,
"Mozilla/5.0 (compatible; oBot/2.3.1; +http://filterdb.iss.net/crawler/)": true,
"Mozilla/5.0 (compatible; OpenHoseBot/2.1; +http://www.openhose.org/bot.html)": true,
"Mozilla/5.0 (compatible; OpsBot/1.0)": true,
"Mozilla/5.0 (compatible; OptimizationCrawler/0.2; +http://www.domainoptima.com/robot)": true,
"Mozilla/5.0 (compatible; PaperLiBot/2.1; http://support.paper.li/entries/20023257-what-is-paper-li)": true,
"Mozilla/5.0 (compatible; PaperLiBot/2.1; https://support.paper.li/hc/en-us/articles/360006695637-PaperLiBot)": true,
"Mozilla/5.0 (compatible; Pinterestbot/1.0; +http://www.pinterest.com/bot.html)": true,
"Mozilla/5.0 (compatible; PrivacyAwareBot/1.1; +http://www.privacyaware.org)": true,
"Mozilla/5.0 (compatible; proximic; +https://www.comscore.com/Web-Crawler)": true,
"Mozilla/5.0 (compatible; Qwantify-dev/1.0; +https://help.qwant.com/bot/)": true,
"Mozilla/5.0 (compatible; Qwantify-dev10230/1.0; +https://help.qwant.com/bot/)": true,
"Mozilla/5.0 (compatible; Qwantify-dev19924/1.0; +https://help.qwant.com/bot/)": true,
"Mozilla/5.0 (compatible; Qwantify-dev19967/1.0; +https://help.qwant.com/bot/)": true,
"Mozilla/5.0 (compatible; Qwantify-dev2270/1.0; +https://help.qwant.com/bot/)": true,
"Mozilla/5.0 (compatible; Qwantify-dev22709/1.0; +https://help.qwant.com/bot/)": true,
"Mozilla/5.0 (compatible; Qwantify-dev31653/1.0; +https://help.qwant.com/bot/)": true,
"Mozilla/5.0 (compatible; Qwantify-dev4541/1.0; +https://help.qwant.com/bot/)": true,
"Mozilla/5.0 (compatible; Qwantify-dev5115/1.0; +https://help.qwant.com/bot/)": true,
"Mozilla/5.0 (compatible; Qwantify-dev846/1.0; +https://help.qwant.com/bot/)": true,
"Mozilla/5.0 (compatible; Qwantify-worker2994/1.0; +https://help.qwant.com/bot/)": true,
"Mozilla/5.0 (compatible; Qwantify/2.2w; +https://www.qwant.com/)/*": true,
"Mozilla/5.0 (compatible; redditbot/1.0; +http://www.reddit.com/feedback)": true,
"Mozilla/5.0 (compatible; rss2tg bot; +http://komar.in/en/rss2tg_crawler)": true,
"Mozilla/5.0 (compatible; Sadakura; +http://shorf.com/bot.php)": true,
"Mozilla/5.0 (compatible; SearchMySiteBot/1.0; +https://searchmysite.net)": true,
"Mozilla/5.0 (compatible; Semanticbot/1.0; +http://sempi.tech/bot.html)": true,
"Mozilla/5.0 (compatible; SemrushBot-BA; +http://www.semrush.com/bot.html)": true,
"Mozilla/5.0 (compatible; SemrushBot-SI/0.97; +http://www.semrush.com/bot.html)": true,
"Mozilla/5.0 (compatible; SemrushBot/1.1~bl; +http://www.semrush.com/bot.html)": true,
"Mozilla/5.0 (compatible; SemrushBot/1~bl; +http://www.semrush.com/bot.html)": true,
"Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)": true,
"Mozilla/5.0 (compatible; SemrushBot; +http://www.semrush.com/bot.html)": true,
"Mozilla/5.0 (compatible; SEOkicks-Robot; +http://www.seokicks.de/robot.html)": true,
"Mozilla/5.0 (compatible; SEOlyticsCrawler/3.0; +http://crawler.seolytics.net/)": true,
"Mozilla/5.0 (compatible; SeznamBot/3.2; +http://fulltext.sblog.cz/)": true,
"Mozilla/5.0 (compatible; SeznamBot/3.2; +http://napoveda.seznam.cz/en/seznambot-intro/)": true,
"Mozilla/5.0 (compatible; SeznamBot/4.0; +http://napoveda.seznam.cz/seznambot-intro/)": true,
"Mozilla/5.0 (compatible; SiteAuditBot/0.97; +http://www.semrush.com/bot.html)": true,
"Mozilla/5.0 (compatible; spbot/4.4.2; +http://OpenLinkProfiler.org/bot )": true,
"Mozilla/5.0 (compatible; spbot/5.0.1; +http://OpenLinkProfiler.org/bot )": true,
"Mozilla/5.0 (compatible; spbot/5.0.2; +http://OpenLinkProfiler.org/bot )": true,
"Mozilla/5.0 (compatible; spbot/5.0; +http://OpenLinkProfiler.org/bot )": true,
"Mozilla/5.0 (compatible; special_archiver/3.1.1 +http://www.archive.org/details/archive.org_bot)": true,
"Mozilla/5.0 (compatible; startmebot/1.0; +https://start.me/bot)": true,
"Mozilla/5.0 (compatible; SurdotlyBot/1.0; +http://sur.ly/bot.html)": true,
"Mozilla/5.0 (compatible; SynapseMediaProxyBot/0.0.1; https://git.pixie.town/f0x/synapse-media-proxy)": true,
"Mozilla/5.0 (compatible; TestCrawler)": true,
"Mozilla/5.0 (compatible; TrendsmapResolver/0.1)": true,
"Mozilla/5.0 (compatible; uMBot-LN/1.0; mailto: crawling@ubermetrics-technologies.com)": true,
"Mozilla/5.0 (compatible; UptimeRobot/2.0; http://www.uptimerobot.com/)": true,
"Mozilla/5.0 (compatible; WebwikiBot/2.1; +https://www.webwiki.de)": true,
"Mozilla/5.0 (compatible; WellKnownBot/0.1; +https://well-known.dev/about/#bot)": true,
"Mozilla/5.0 (compatible; woriobot +http://worio.com)": true,
"Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve.k8s.crawler-production/1688317128-0": true,
"Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)": true,
"Mozilla/5.0 (compatible; YaK/1.0; http://linkfluence.com/; bot@linkfluence.com)": true,
"Mozilla/5.0 (compatible; YandexAccessibilityBot/3.0; +http://yandex.com/bots) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0": true,
"Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0": true,
"Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)": true,
"Mozilla/5.0 (compatible; YandexFavicons/1.0; +http://yandex.com/bots)": true,
"Mozilla/5.0 (compatible; YandexImages/3.0; +http://yandex.com/bots)": true,
"Mozilla/5.0 (compatible; YandexRenderResourcesBot/1.0; +http://yandex.com/bots) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0": true,
"Mozilla/5.0 (compatible; YandexUserproxy; robot; +http://yandex.com/bots)": true,
"Mozilla/5.0 (compatible; Yeti/1.1; +http://help.naver.com/robots/)": true,
"Mozilla/5.0 (compatible; YetiBOT/0.1b; +http://yetibot.ovh)": true,
"Mozilla/5.0 (compatible; zitebot support [at] zite [dot] com +http://zite.com)": true,
"Mozilla/5.0 (compatible;PetalBot;+https://webmaster.petalsearch.com/site/petalbot)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 10_3 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.75 Mobile/14E5239e YisouSpider/5.0 Safari/602.1": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 15_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 Weibo (iPhone14,3__weibo__13.6.3__iphone__os15.1)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 15_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 Weibo (iPhone12,8__weibo__13.6.3__iphone__os15.6)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Applebot/0.3; +http://www.apple.com/go/applebot)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; SiteAuditBot/0.97; +http://www.semrush.com/bot.html)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4 (compatible; Laserlikebot/0.1)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)": true,
"Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13F69 Safari/601.1": true,
"Mozilla/5.0 (Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0": true,
"Mozilla/5.0 (Linux x86_64; rv:114.0) Gecko/20100101 Firefox/114.0": true,
"Mozilla/5.0 (Linux; Android 10; BAH3-W59 Build/HUAWEIBAH3-W59; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/92.0.4515.105 Safari/537.36 Weibo (HUAWEI-BAH3-W59__weibo__12.5.2__android__android10)": true,
"Mozilla/5.0 (Linux; Android 10; VCE-AL00 Build/HUAWEIVCE-AL00; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/92.0.4515.105 Mobile Safari/537.36 Weibo (HUAWEI-VCE-AL00__weibo__13.6.3__android__android10)": true,
"Mozilla/5.0 (Linux; Android 13; V2244A Build/TP1A.220624.014; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/101.0.4951.74 Mobile Safari/537.36V2244A_13_WeiboIntlAndroid_6200": true,
"Mozilla/5.0 (Linux; Android 13; V2302A Build/TP1A.220624.014; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/101.0.4951.74 Mobile Safari/537.36 V2302A_13_WeiboIntlAndroid_2612": true,
"Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; https://zhanzhang.toutiao.com/)": true,
"Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)": true,
"Mozilla/5.0 (Linux; Android 5.1.1; A37f) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.74 Mobile Safari/537.36": true,
"Mozilla/5.0 (Linux; Android 6.0.1; vivo 1606 Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.84 Mobile Safari/537.36 VivoBrowser/5.7.0.6": true,
"Mozilla/5.0 (Linux; Android 6.0.1; vivo Y66 Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/87.0.4280.141 Mobile Safari/537.36 VivoBrowser/10.9.14.0": true,
"Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://webmaster.petalsearch.com/site/petalbot)": true,
"Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko) Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:42.0) Gecko/20100101 Firefox/42.0": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:49.0) Gecko/20100101 Firefox/49.0 (FlipboardProxy/1.2; +http://flipboard.com/browserproxy)": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0 (FlipboardProxy/1.1; +http://flipboard.com/browserproxy)": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:29.0) Gecko/20100101 Firefox/29.0": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.7 DuckDuckGo/7 Safari/605.1.15": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.1 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot)": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.81 Safari/537.36 Edg/104.0.1293.54; Bot": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 webtru_crawler": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 AppEngine-Google; (+http://code.google.com/appengine; appid: s~feedly-social)": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36": true,
"Mozilla/5.0 (Macintosh; Intel Mac OS X 13_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 ISSCyberRiskCrawler/1.1.3": true,
"Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5": true,
"Mozilla/5.0 (Morningscore Bot/1.0)": true,
"Mozilla/5.0 (TweetmemeBot/4.0; +http://datasift.com/bot.html) Gecko/20100101 Firefox/31.0": true,
"Mozilla/5.0 (Windows; Crawler; U; Windows NT 6.0; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)": true,
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.5735.179 Safari/537.36 (compatible; Google-Safety; +http://www.google.com/bot.html)": true,
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36 (compatible; Google-Safety; +http://www.google.com/bot.html)": true,
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; trendictionbot0.5.0; trendiction search; http://www.trendiction.de/bot; please let us know of any problems; web at trendiction.com) Gecko/20170101 Firefox/67.0": true,
"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36": true,
"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0": true,
"Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko": true,
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/534.27+ (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27": true,
"Mozilla/5.0 (Windows NT 5.1; rv:33.0) Gecko/20100101 Firefox/33.0": true,
"Mozilla/5.0 (Windows NT 5.1; rv:6.0.2) Gecko/20100101 Firefox/6.0.2": true,
"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1; 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)": true,
"Mozilla/5.0 (Windows NT 6.1) (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot)": true,
"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1; 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)": true,
"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.81 YisouSpider/5.0 Safari/537.36": true,
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36": true,
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137 Safari/537.36": true,
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36": true,
@@ -151,10 +283,11 @@ var bots = map[string]bool{
"Mozilla/5.0 (Windows NT 6.2; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0": true,
"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36": true,
"Mozilla/5.0 (Windows NT 6.3; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0": true,
"Mozilla/5.0 (Windows; Crawler; U; Windows NT 6.0; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)": true,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:1.6) Gecko/20040206 Firefox/0.8": true,
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en; rv:1.9.0.13) Gecko/2009073022 Firefox/3.5.2 (.NET CLR 3.5.30729) SurveyBot/2.3 (DomainTools)": true,
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36 HubSpot Webcrawler": true,
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4": true,
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en; rv:1.9.0.13) Gecko/2009073022 Firefox/3.5.2 (.NET CLR 3.5.30729) SurveyBot/2.3 (DomainTools)": true,
"Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10": true,
"Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.0; trendictionbot0.5.0; trendiction search; http://www.trendiction.de/bot; please let us know of any problems; web at trendiction.com) Gecko/20071127 Firefox/3.0.0.11": true,
"Mozilla/5.0 (Windows; U; WinNT4.0; de-DE; rv:1.7.6) Gecko/20050226 Firefox/1.0.1": true,
@@ -163,63 +296,132 @@ var bots = map[string]bool{
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.155 Safari/537.36/Gringe": true,
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36": true,
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36": true,
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.8) Gecko/20061201 Firefox/2.0.0.8": true,
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/61.0.3163.100 Chrome/61.0.3163.100 Safari/537.36 PingdomPageSpeed/1.0 (pingbot/2.0; +http://www.pingdom.com/)": true,
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8) Gecko/20060118 Firefox/1.5": true,
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.8) Gecko/20061201 Firefox/2.0.0.8": true,
"Mozilla/5.0 (X11; U; Linux i686; fr-FR; rv:1.8.1.6) Gecko/20080208 Ubuntu/7.10 (gutsy) Firefox/2.0.0.12": true,
"Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.9) Gecko/2009042113 Ubuntu/8.04 (hardy) Firefox/3.0.9": true,
"Mozilla/5.0 (X11; U; Linux i686; hu-HU; rv:1.9.1.9) Gecko/20100330 Fedora/3.5.9-1.fc12 Firefox/3.5.9": true,
"Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.9) Gecko/2008061015 Firefox/3.0": true,
"Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6": true,
"Mozilla/5.0 (X11; U; Linux x86_64; cs-CZ; rv:1.9.1.7) Gecko/20100106 Ubuntu/9.10 (karmic) Firefox/3.5.7": true,
"Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.2.8) Googlebot-Compatible Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8": true,
"Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.18) Gecko/20081112 Fedora/2.0.0.18-1.fc8 Firefox/2.0.0.18": true,
"Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.3) Gecko/20090914 Slackware/13.0_stable Firefox/3.5.3": true,
"Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9a1) Gecko/20060112 Firefox/1.6a1": true,
"Mozilla/5.0 (X11; U; Linux x86_64; zh-TW; rv:1.9.0.13) Gecko/2009080315 Ubuntu/9.04 (jaunty) Firefox/3.0.13": true,
"myspider/Nutch-1.10": true,
"netEstate NE Crawler (+http://www.website-datenbank.de/)": true,
"omgili/0.5 +http://omgili.com": true,
"PercolateCrawler/4 (ops@percolate.com)": true,
"python-requests/2.6.2 CPython/2.7.6 Linux/3.13.0-43-generic": true,
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:15.0) Xing Bot": true,
"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot": true,
"Mozilla/5.0(compatible;Googlebot/2.1; +http://www.google.com/bot.html)": true,
"Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/)": true,
"Mozilla/5.0/(compatible; heritrix/3.3.0-SNAPSHOT-20150803-2130 +http://literatur-im-netz.dla-marbach.de)": true,
"Mozilla/5.0/Chrome/12.0.742.112 (Macintosh; Intel Mac OS X 10_6_8)": true,
"Mozilla/5.0/Firefox/42.0 (contactbigdatafr at gmail.com)": true,
"Mozilla/9.0 (compatible; Staddlebot/1.0)": true,
"myspider/Nutch-1.10": true,
"navigation": true,
"netEstate NE Crawler (+http://www.website-datenbank.de/)": true,
"Netvibes (crawler; http://www.netvibes.com)": true,
"Netvibes (crawler; https://www.netvibes.com)": true,
"Nextcloud Server Crawler": true,
"omgili/0.5 +http://omgili.com": true,
"OWLer/0.1 (built with StormCrawler; https://ows.eu/owler; owl@ow-s.eu)": true,
"PercolateCrawler/4 (ops@percolate.com)": true,
"PhxBot/0.1 (phxbot@protonmail.com)": true,
"PlayStore-Google Mozilla/5.0 Chrome/114.0.5735.179 (KHTML, like Gecko; compatible; +http://www.google.com/bot.html)": true,
"PlurkBot; Mozilla/5.0 (X11; Linux x86_64; rv:85.0) Gecko/20100101 Firefox/85.0(facebookexternalhitTwitterbot compatible; PlurkBot; +https://www.plurk.com/)": true,
"PubMatic Crawler Bot": true,
"PulsePoint-Ads.txt-Crawler/1.1": true,
"python-requests/2.6.2 CPython/2.7.6 Linux/3.13.0-61-generic": true,
"Python-urllib/2.7": true,
"Quora-Bot/1.0 (http://www.quora.com)": true,
"R6_CommentReader(www.radian6.com/crawler)": true,
"R6_FeedFetcher(www.radian6.com/crawler)": true,
"RandomWebSiteBot 1.0": true,
"read.write.as": true,
"Readybot.io (https://readybot.io)": true,
"repology-linkchecker/1 (+https://repology.org/docs/bots)": true,
"rogerbot/1.0 (http://moz.com/help/pro/what-is-rogerbot-, rogerbot-crawler+shiny@moz.com)": true,
"Ruby": true,
"SafeDNSBot (https://www.safedns.com/searchbot)": true,
"SafeDNS search bot/Nutch-1.9 (https://www.safedns.com/searchbot; support [at] safedns [dot] com)": true,
"SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)": true,
"Screaming Frog SEO Spider/4.1": true,
"Screaming Frog SEO Spider/5.0": true,
"Screaming Frog SEO Spider/5.1": true,
"semanticbot": true,
"semanticbot (info@semanticaudience.com)": true,
"ShowyouBot (http://showyou.com/crawler)": true,
"Slackbot 1.0 (+https://api.slack.com/robots)": true,
"Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)": true,
"Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)": true,
"SafeDNS search bot/Nutch-1.9 (https://www.safedns.com/searchbot; support [at] safedns [dot] com)": true,
"SafeDNSBot (https://www.safedns.com/searchbot)": true,
"Sellers.Guide Crawler by Primis": true,
"semanticbot": true,
"SEMrushBot": true,
"SeRanking SEOChecker": true,
"SerendeputyBot/0.8.6 (http://serendeputy.com/about/serendeputy-bot)": true,
"serpstatbot/2.1 (advanced backlink tracking bot; https://serpstatbot.com/; abuse@serpstatbot.com)": true,
"ShowyouBot (http://showyou.com/crawler)": true,
"SiteCheckerBotCrawler/1.0 (+http://sitechecker.pro)": true,
"Slack-ImgProxy (+https://api.slack.com/robots)": true,
"Slackbot 1.0 (+https://api.slack.com/robots)": true,
"Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)": true,
"Slzji.com Search Bot (https://www.slzji.com/search_bot)": true,
"Snap URL Preview Service; bot; snapchat; https://developers.snap.com/robots": true,
"Snap-URL-Preview (bot; snapchat; +https://developers.snap.com/robots)": true,
"Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)": true,
"spider": true,
"spiderbot": true,
"Stratagems Kumo": true,
"Traackr.com Bot": true,
"voltron": true,
"Wotbox/2.01 (+http://www.wotbox.com/bot/)": true,
"yacybot (freeworld/global; amd64 Linux 3.12.43-52.6-default; java 1.8.0_40; Europe/en) http://yacy.net/bot.html": true,
"yacybot (/global; x86_64 Mac OS X 10.11.4; java 1.8.0_77; America/en) http://yacy.net/bot.html": true,
"YisouSpider": true,
"Superfeedr bot/2.0 http://superfeedr.com - Make your feeds realtime: get in touch - feed-id:1396250262": true,
"Synapse (bot; +https://github.com/matrix-org/synapse)": true,
"TelegramBot (like TwitterBot)": true,
"TheFeedReaderBot/2.0": true,
"tiny.write.as": true,
"Traackr.com Bot": true,
"Twingly Recon": true,
"Twitterbot": true,
"voltron": true,
"webprosbot/2.0 (+mailto:abuse-6337@webpros.com)": true,
"webscraper": true,
"Who.is Bot": true,
"Wotbox/2.01 (+http://www.wotbox.com/bot/)": true,
"wp.com feedbot/1.0 (+https://wp.com)": true,
"Write.as v1.7.0; Android": true,
"write.as": true,
"WriteFreely.org Crawler (https://writefreely.org/instances)": true,
"Y!J-ASR/0.1 crawler (http://www.yahoo-help.jp/app/answers/detail/p/595/a_id/42716/)": true,
"YisouSpider": true,
"ZoominfoBot (zoominfobot at zoominfo dot com)": true,
}
var botPrefixes = []string{
"AdsTxtCrawler/",
"Akkoma ",
"Apache-HttpClient/",
"bitlybot/",
"curl/",
"DuckDuckBot/",
"facebookexternalhit/",
"FediDB/",
"Friendica",
"Googlebot-Image/",
"Googlebot/",
"hackney/",
"http.rb/",
"kbinBot ",
"libwww-perl/",
"Mediumbot-MetaTagFetcher/",
"Mozilla/5.0 (compatible; AhrefsBot/",
"Mozilla/5.0 (compatible; Applebot/",
"Mozilla/5.0 (compatible; archive.org_bot",
"PHP/",
"TelegramBot",
"Twitterbot/",
"hackney/",
"http.rb/",
"PixelFedBot/",
"Pleroma ",
"python-requests/",
"Python-urllib/",
"RepoLookoutBot/",
"Screaming Frog SEO Spider/",
"semanticbot ",
"SummalyBot/",
"TelegramBot",
"Twingly Recon-",
"Twitterbot/",
"WhatsApp/",
"yacybot ",
}
var botPhrases = []string{
"bingbot",
"Googlebot",
}
// IsBot returns whether or not the provided User-Agent string is a known bot
@@ -236,5 +438,10 @@ func IsBot(ua string) bool {
return true
}
}
for _, p := range botPhrases {
if strings.Contains(ua, p) {
return true
}
}
return false
}
bots/bots_test.go
+3 -1
View File
@@ -4,7 +4,7 @@ import "testing"
func TestIsBot(t *testing.T) {
tests := map[string]bool{
"Twitterbot/1.0": true,
"Twitterbot/1.0": true,
"http.rb/2.2.2 (Mastodon/1.6.0; +https://insolente.im/)": true,
"http.rb/2.2.2 (Mastodon/1.5.1; +https://mastodon.cloud/)": true,
"http.rb/2.2.2 (Mastodon/1.6.0rc5; +https://mastodon.sdf.org/)": true,
@@ -13,6 +13,8 @@ func TestIsBot(t *testing.T) {
"Mozilla/5.0 (compatible; Applebot/0.3; +http://www.apple.com/go/applebot)": true,
"Mozilla/5.0 (compatible; archive.org_bot +http://www.archive.org/details/archive.org_bot)": true,
"Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)": true,
"Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)": true,
}
for ua, r := range tests {
category/category.go
+52
View File
@@ -0,0 +1,52 @@
// Package category supports post categories
package category
import (
"errors"
"github.com/writeas/slug"
)
var (
ErrNotFound = errors.New("category doesn't exist")
)
// Category represents a post tag with additional metadata, like a title and slug.
type Category struct {
ID int64 `json:"-"`
Hashtag string `json:"hashtag"`
Slug string `json:"slug"`
Title string `json:"title"`
PostCount int64 `json:"post_count"`
}
// NewCategory creates a Category you can insert into the database, based on a hashtag. It automatically breaks up the
// hashtag by words, based on capitalization, for both the title and a URL-friendly slug.
func NewCategory(hashtag string) *Category {
title := titleFromHashtag(hashtag)
return &Category{
Hashtag: hashtag,
Slug: slug.Make(title),
Title: title,
}
}
// NewCategoryFromPartial creates a Category from a partially-populated Category, such as when a user initially creates
// one.
func NewCategoryFromPartial(cat *Category) *Category {
newCat := &Category{
Hashtag: cat.Hashtag,
}
// Create title from hashtag, if none supplied
if cat.Title == "" {
newCat.Title = titleFromHashtag(cat.Hashtag)
} else {
newCat.Title = cat.Title
}
// Create slug from title, if none supplied; otherwise ensure slug is valid
if cat.Slug == "" {
newCat.Slug = slug.Make(newCat.Title)
} else {
newCat.Slug = slug.Make(cat.Slug)
}
return newCat
}
category/tags.go
+47
View File
@@ -0,0 +1,47 @@
package category
import (
"strings"
"unicode"
)
// titleFromHashtag generates an all-lowercase title, with spaces inserted based on initial capitalization -- e.g.
// "MyWordyTag" becomes "my wordy tag".
func titleFromHashtag(hashtag string) string {
var t strings.Builder
var prev rune
for i, c := range hashtag {
if unicode.IsUpper(c) {
if i > 0 && !unicode.IsUpper(prev) {
// Insert space if previous rune wasn't also uppercase (e.g. an abbreviation)
t.WriteRune(' ')
}
t.WriteRune(unicode.ToLower(c))
} else {
t.WriteRune(c)
}
prev = c
}
return t.String()
}
// HashtagFromTitle generates a valid single-word, camelCase hashtag from a title (which might include spaces,
// punctuation, etc.).
func HashtagFromTitle(title string) string {
var t strings.Builder
var prev rune
for _, c := range title {
if !unicode.IsLetter(c) && !unicode.IsNumber(c) {
prev = c
continue
}
if unicode.IsSpace(prev) {
// Uppercase next word
t.WriteRune(unicode.ToUpper(c))
} else {
t.WriteRune(c)
}
prev = c
}
return t.String()
}
category/tags_test.go
+60
View File
@@ -0,0 +1,60 @@
package category
import "testing"
func TestTitleFromHashtag(t *testing.T) {
tests := []struct {
name string
hashtag string
expTitle string
}{
{"proper noun", "Jane", "jane"},
{"full name", "JaneDoe", "jane doe"},
{"us words", "unitedStates", "united states"},
{"usa", "USA", "usa"},
{"us monoword", "unitedstates", "unitedstates"},
{"100dto", "100DaysToOffload", "100 days to offload"},
{"iphone", "iPhone", "iphone"},
{"ilike", "iLikeThis", "i like this"},
{"abird", "aBird", "a bird"},
{"all caps", "URGENT", "urgent"},
{"smartphone", "スマートフォン", "スマートフォン"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
res := titleFromHashtag(test.hashtag)
if res != test.expTitle {
t.Fatalf("#%s: got '%s' expected '%s'", test.hashtag, res, test.expTitle)
}
})
}
}
func TestHashtagFromTitle(t *testing.T) {
tests := []struct {
name string
title string
expHashtag string
}{
{"proper noun", "Jane", "Jane"},
{"full name", "Jane Doe", "JaneDoe"},
{"us upper words", "United States", "UnitedStates"},
{"us lower words", "united states", "unitedStates"},
{"usa", "USA", "USA"},
{"100dto", "100 Days To Offload", "100DaysToOffload"},
{"iphone", "iPhone", "iPhone"},
{"ilike", "I like this", "ILikeThis"},
{"abird", "a Bird", "aBird"},
{"all caps", "URGENT", "URGENT"},
{"punctuation", "John’s Stories", "JohnsStories"},
{"smartphone", "スマートフォン", "スマートフォン"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
res := HashtagFromTitle(test.title)
if res != test.expHashtag {
t.Fatalf("%s: got '%s' expected '%s'", test.title, res, test.expHashtag)
}
})
}
}
go.mod
+8 -6
View File
@@ -4,14 +4,16 @@ go 1.10
require (
github.com/gofrs/uuid v3.3.0+incompatible
github.com/kr/pretty v0.1.0 // indirect
github.com/kylemcc/twitter-text-go v0.0.0-20180726194232-7f582f6736ec
github.com/microcosm-cc/bluemonday v1.0.2
github.com/microcosm-cc/bluemonday v1.0.23
github.com/rainycape/unidecode v0.0.0-20150907023854-cb7f23ec59be // indirect
github.com/shurcooL/sanitized_anchor_name v1.0.0 // indirect
github.com/writeas/impart v1.1.0
github.com/writeas/go-strip-markdown/v2 v2.1.1
github.com/writeas/impart v1.1.1
github.com/writeas/openssl-go v1.0.0
github.com/writeas/saturday v1.6.0
golang.org/x/crypto v0.0.0-20190131182504-b8fe1690c613
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 // indirect
github.com/writeas/saturday v1.7.1
github.com/writeas/slug v1.2.0
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
gopkg.in/yaml.v1 v1.0.0-20140924161607-9f9df34309c0 // indirect
)
go.sum
+56 -15
View File
@@ -1,28 +1,69 @@
github.com/gofrs/uuid v1.2.0 h1:coDhrjgyJaglxSjxuJdqQSSdUpG3w6p1OwN2od6frBU=
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
github.com/gofrs/uuid v3.3.0+incompatible h1:8K4tyRfvU1CYPgJsveYFQMhpFd/wXNM7iK6rR7UHz84=
github.com/gofrs/uuid v3.3.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/gorilla/css v1.0.0 h1:BQqNyPTi50JCFMTw/b67hByjMVXZRwGha6wxVGkeihY=
github.com/gorilla/css v1.0.0/go.mod h1:Dn721qIggHpt4+EFCcTLTU/vk5ySda2ReITrtgBl60c=
github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kylemcc/twitter-text-go v0.0.0-20180726194232-7f582f6736ec h1:ZXWuspqypleMuJy4bzYEqlMhJnGAYpLrWe5p7W3CdvI=
github.com/kylemcc/twitter-text-go v0.0.0-20180726194232-7f582f6736ec/go.mod h1:voECJzdraJmolzPBgL9Z7ANwXf4oMXaTCsIkdiPpR/g=
github.com/microcosm-cc/bluemonday v1.0.2 h1:5lPfLTTAvAbtS0VqT+94yOtFnGfUWYyx0+iToC3Os3s=
github.com/microcosm-cc/bluemonday v1.0.2/go.mod h1:iVP4YcDBq+n/5fb23BhYFvIMq/leAFZyRl6bYmGDlGc=
github.com/microcosm-cc/bluemonday v1.0.23 h1:SMZe2IGa0NuHvnVNAZ+6B38gsTbi5e4sViiWJyDDqFY=
github.com/microcosm-cc/bluemonday v1.0.23/go.mod h1:mN70sk7UkkF8TUr2IGBpNN0jAgStuPzlK76QuruE/z4=
github.com/rainycape/unidecode v0.0.0-20150907023854-cb7f23ec59be h1:ta7tUOvsPHVHGom5hKW5VXNc2xZIkfCKP8iaqOyYtUQ=
github.com/rainycape/unidecode v0.0.0-20150907023854-cb7f23ec59be/go.mod h1:MIDFMn7db1kT65GmV94GzpX9Qdi7N/pQlwb+AN8wh+Q=
github.com/shurcooL/sanitized_anchor_name v1.0.0 h1:PdmoCO6wvbs+7yrJyMORt4/BmY5IYyJwS/kOiWx8mHo=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/writeas/impart v1.1.0 h1:nPnoO211VscNkp/gnzir5UwCDEvdHThL5uELU60NFSE=
github.com/writeas/impart v1.1.0/go.mod h1:g0MpxdnTOHHrl+Ca/2oMXUHJ0PcRAEWtkCzYCJUXC9Y=
github.com/writeas/go-strip-markdown/v2 v2.1.1 h1:hAxUM21Uhznf/FnbVGiJciqzska6iLei22Ijc3q2e28=
github.com/writeas/go-strip-markdown/v2 v2.1.1/go.mod h1:UvvgPJgn1vvN8nWuE5e7v/+qmDu3BSVnKAB6Gl7hFzA=
github.com/writeas/impart v1.1.1 h1:RyA9+CqbdbDuz53k+nXCWUY+NlEkdyw6+nWanxSBl5o=
github.com/writeas/impart v1.1.1/go.mod h1:g0MpxdnTOHHrl+Ca/2oMXUHJ0PcRAEWtkCzYCJUXC9Y=
github.com/writeas/openssl-go v1.0.0 h1:YXM1tDXeYOlTyJjoMlYLQH1xOloUimSR1WMF8kjFc5o=
github.com/writeas/openssl-go v1.0.0/go.mod h1:WsKeK5jYl0B5y8ggOmtVjbmb+3rEGqSD25TppjJnETA=
github.com/writeas/saturday v1.6.0 h1:HNUtX8TVJJnSdxE8vaAgtHiAJVt+Of5AJYlm62pmVlI=
github.com/writeas/saturday v1.6.0/go.mod h1:ETE1EK6ogxptJpAgUbcJD0prAtX48bSloie80+tvnzQ=
golang.org/x/crypto v0.0.0-20190131182504-b8fe1690c613 h1:MQ/ZZiDsUapFFiMS+vzwXkCTeEKaum+Do5rINYJDmxc=
golang.org/x/crypto v0.0.0-20190131182504-b8fe1690c613/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/net v0.0.0-20181220203305-927f97764cc3 h1:eH6Eip3UpmR+yM/qI9Ijluzb1bNv/cAU/n+6l8tRSis=
golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
github.com/writeas/saturday v1.7.1 h1:lYo1EH6CYyrFObQoA9RNWHVlpZA5iYL5Opxo7PYAnZE=
github.com/writeas/saturday v1.7.1/go.mod h1:ETE1EK6ogxptJpAgUbcJD0prAtX48bSloie80+tvnzQ=
github.com/writeas/slug v1.2.0 h1:EMQ+cwLiOcA6EtFwUgyw3Ge18x9uflUnOnR6bp/J+/g=
github.com/writeas/slug v1.2.0/go.mod h1:RE8shOqQP3YhsfsQe0L3RnuejfQ4Mk+JjY5YJQFubfQ=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 h1:7I4JAnoQBe7ZtJcBaYHi5UtiO8tQHbUSXxL+pnGRANg=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.8.0 h1:Zrh2ngAOFYneWTAIAPethzeaQLuHwhuBkuV6ZiRnUaQ=
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v1 v1.0.0-20140924161607-9f9df34309c0 h1:POO/ycCATvegFmVuPpQzZFJ+pGZeX22Ufu6fibxDVjU=
gopkg.in/yaml.v1 v1.0.0-20140924161607-9f9df34309c0/go.mod h1:WDnlLJ4WF5VGsH/HVa3CI79GS0ol3YnhVnKP89i0kNg=
id/random.go
+13 -1
View File
@@ -1,8 +1,8 @@
package id
import (
"fmt"
"crypto/rand"
"fmt"
)
// GenerateRandomString creates a random string of characters of the given
@@ -24,3 +24,15 @@ func GenerateRandomString(dictionary string, l int) string {
func GenSafeUniqueSlug(slug string) string {
return fmt.Sprintf("%s-%s", slug, GenerateRandomString("0123456789bcdfghjklmnpqrstvwxyz", 4))
}
// Generate62RandomString creates a random string with the given length
// consisting of characters in [A-Za-z0-9].
func Generate62RandomString(l int) string {
return GenerateRandomString("0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz", l)
}
// GenerateFriendlyRandomString creates a random string of characters with the
// given length consisting of characters in [a-z0-9].
func GenerateFriendlyRandomString(l int) string {
return GenerateRandomString("0123456789abcdefghijklmnopqrstuvwxyz", l)
}
l10n/phrases.go
+13 -7
View File
@@ -14,23 +14,29 @@ var phrases = map[string]string{
"Blogs": "Blogs",
"Enter": "Enter",
"Newer": "Newer",
"Next": "Next",
"Older": "Older",
"Posts": "Posts",
"Previous": "Previous",
"Publish to...": "Publish to...",
"Publish": "Publish",
"Read more...": "Read more...",
"Subscribe": "Subscribe",
"This blog requires a password.": "This blog requires a password.",
"Toggle theme": "Toggle theme",
"View posts": "View Posts",
"delete": "delete",
"edit": "edit",
"email subscription confirm": "Please check your email and click the confirmation link to subscribe.",
"email subscription prompt": "Enter your email to subscribe to updates.",
"email subscription success": "Subscribed. You'll now receive future blog posts via email.",
"move to...": "move to...",
"pin": "pin",
"published with write.as": "published with write.as",
"share modal ending": "Send it to a friend, share it across the web, or maybe tweet it. Learn more.",
"share modal introduction": "Each published post has a secret, unique URL you can share with anyone. This is that URL:",
"share modal title": "Share this post",
"share": "share",
"unpin": "unpin",
"title dash": "&mdash;",
"published with write.as": "published with write.as",
"share modal ending": "Send it to a friend, share it across the web, or maybe tweet it. Learn more.",
"share modal introduction": "Each published post has a secret, unique URL you can share with anyone. This is that URL:",
"share modal title": "Share this post",
"share": "share",
"unpin": "unpin",
"title dash": "&mdash;",
}
l10n/phrases_ar.go
+8 -6
View File
@@ -10,17 +10,19 @@ var phrasesAR = map[string]string{
"Publish to...": "النشر إلى…",
"Publish": "نشر",
"Read more...": "اقرأ المزيد…",
"Subscribe": "اشترك",
"This blog requires a password.": "هذه المدونة تتطلب کلمة سرية.",
"Toggle theme": "تغيير القالب",
"View posts": "مشاهدة المنشورات",
"delete": "حذف",
"edit": "تعديل",
"email subscription prompt": "ادخل عنوان بريدك الإلكتروني للإشتراك في التحديثات",
"move to...": "أنقله إلى…",
"pin": "تدبيس",
"published with write.as": "مدعوم من write.as",
"share modal ending": "ارسلهه إلى صديق، شاركه عبر الويب، أو غرّده. تعلّم المزيد.",
"share modal introduction": "كل الفيديوهات المنشورة لديها رابط سري مميز، يمكنك مشاركته. هذا هو الرابط:",
"share modal title": "شارك هذا المنشور",
"share": "شارك",
"unpin": "إلغاء التدبيس",
"published with write.as": "مدعوم من write.as",
"share modal ending": "ارسلهه إلى صديق، شاركه عبر الويب، أو غرّده. تعلّم المزيد.",
"share modal introduction": "كل الفيديوهات المنشورة لديها رابط سري مميز، يمكنك مشاركته. هذا هو الرابط:",
"share modal title": "شارك هذا المنشور",
"share": "شارك",
"unpin": "إلغاء التدبيس",
}
l10n/phrases_cs.go
+28
View File
@@ -0,0 +1,28 @@
package l10n
var phrasesCS = map[string]string{
"Anonymous post": "Anonymní článek",
"Blogs": "Blogy",
"Enter": "Vstoupit",
"Newer": "Novější",
"Older": "Starší",
"Posts": "Články",
"Publish to...": "Zveřejnit do...",
"Publish": "Zveřejnit",
"Read more...": "Číst dále...",
"Subscribe": "Odebírat",
"This blog requires a password.": "Tento blog je zaheslován.",
"Toggle theme": "Přepnout vzhled",
"View posts": "Zobrazit články",
"delete": "smazat",
"edit": "upravit",
"email subscription prompt": "Zadejte svůj e-mail a přihlaste se k odběru aktualizací.",
"move to...": "přesunout do...",
"pin": "připnout",
"published with write.as": "publikováno pomocí write.as",
"share modal ending": "Pošlete ji kamarádům, sdílejte na webu nebo na sociálních sítích. Zjistit více.",
"share modal introduction": "Každý zveřejněný článek má tajnou unikátní adresu, kterou můžete komukoliv poslat. Unikátní adresa pro tento článek je:",
"share modal title": "Sdílet tento článek",
"share": "sdílet",
"unpin": "odepnout",
}
l10n/phrases_da.go
+28
View File
@@ -0,0 +1,28 @@
package l10n
var phrasesDA = map[string]string{
"Anonymous post": "Anonymt indlæg",
"Blogs": "Blogs",
"Enter": "Enter",
"Newer": "Nyere",
"Older": "Ældre",
"Posts": "Indlæg",
"Publish to...": "Udgiv på...",
"Publish": "Udgiv",
"Read more...": "Læs videre...",
"Subscribe": "Abonnér",
"This blog requires a password.": "Den her blog kræver en adgangskode.",
"Toggle theme": "Skift design",
"View posts": "Vis indlæg",
"delete": "slet",
"edit": "redigér",
"email subscription prompt": "Indtast din email for at få tilsendt opdateringer.",
"move to...": "flyt til...",
"pin": "sæt øverst",
"published with write.as": "udgivet med write.as",
"share modal ending": "Send det til en ven, del det på nettet, eller lav et tweet. Læs mere.",
"share modal introduction": "Hvert indlæg har sit eget, hemmelige link, som du kan dele med enhver. Her er det førnævnte link:",
"share modal title": "Del indlæg",
"share": "del",
"unpin": "fjern øverst",
}
l10n/phrases_de.go
+10 -8
View File
@@ -7,21 +7,23 @@ var phrasesDE = map[string]string{
"Newer": "Neuer",
"Older": "Älter",
"Posts": "Beiträge",
"Publish to...": "Veröffentlichen zu...",
"Publish to...": "Veröffentlichen zu",
"Publish": "Veröffentlichen",
"Read more...": "Weiterlesen...",
"Subscribe": "Abonnieren",
"This blog requires a password.": "Dieser Blog benötigt ein Passwort",
"Toggle theme": "Design ändern",
"View posts": "Beiträge ansehen",
"delete": "Löschen",
"edit": "Bearbeiten",
"email subscription prompt": "Geben Sie Ihre E-Mail-Adresse ein, um Updates zu abonnieren.",
"move to...": "Verschieben nach...",
"pin": "Anheften",
"published with write.as": "veröffentlicht mit write.as",
"share modal ending": "Schicke es Freunden, teile es im Netz oder vielleicht tweete es. Lerne mehr.",
"share modal introduction": "Jeder veröffentlichte Beitrag hat eine geheime, einmalige URL, die du mit jedem teilen kannst. Die URL ist:",
"share modal title": "Teile diesen Beitrag",
"share": "Teilen",
"unpin": "Lösen",
"title dash": "&ndash;",
"published with write.as": "Veröffentlicht mit write.as",
"share modal ending": "Schicke es Freunden, teile es im Netz oder vielleicht tweete es. Lerne mehr.",
"share modal introduction": "Jeder veröffentlichte Beitrag hat eine geheime, einmalige URL, die du mit jedem teilen kannst. Die URL ist:",
"share modal title": "Teile diesen Beitrag",
"share": "Teilen",
"unpin": "Lösen",
"title dash": "&ndash;",
}
l10n/phrases_el.go
+8 -6
View File
@@ -10,17 +10,19 @@ var phrasesEL = map[string]string{
"Publish to...": "Δημοσίευση στο...",
"Publish": "Δημοσίευση",
"Read more...": "Διαβάστε περισσότερα...",
"Subscribe": "Εγγραφείτε",
"This blog requires a password.": "Αυτό το ιστολόγιο απαιτεί κωδικό.",
"Toggle theme": "Αλλαγή θέματος",
"View posts": "Προβολή Δημοσιεύσεων",
"delete": "διαγραφή",
"edit": "επεξεργασία",
"email subscription prompt": "Εισάγετε το email σας για να εγγραφείτε στις ενημερώσεις.",
"move to...": "μετακίνηση στο...",
"pin": "καρφίτσωμα",
"published with write.as": "δημοσιεύθηκε με το write.as",
"share modal ending": "Στείλτε το σε έναν φίλο, μοιραστείτε το στο διαδίκτυο ή κάντε το tweet. Μάθετε περισσότερα.",
"share modal introduction": "Κάθε αναρτημένη δημοσίευση έχει ένα κρυφό, μοναδικό σύνδεσμο που μπορείτε να μοιραστείτε με οποιονδήποτε. Αυτός είναι ο εν λόγω σύνδεσμος:",
"share modal title": "Μοιραστείτε αυτή τη δημοσίευση",
"share": "διαμοιρασμός",
"unpin": "ξεκαρφίτσωμα",
"published with write.as": "δημοσιεύθηκε με το write.as",
"share modal ending": "Στείλτε το σε έναν φίλο, μοιραστείτε το στο διαδίκτυο ή κάντε το tweet. Μάθετε περισσότερα.",
"share modal introduction": "Κάθε αναρτημένη δημοσίευση έχει ένα κρυφό, μοναδικό σύνδεσμο που μπορείτε να μοιραστείτε με οποιονδήποτε. Αυτός είναι ο εν λόγω σύνδεσμος:",
"share modal title": "Μοιραστείτε αυτή τη δημοσίευση",
"share": "διαμοιρασμός",
"unpin": "ξεκαρφίτσωμα",
}
l10n/phrases_eo.go
+6 -6
View File
@@ -17,10 +17,10 @@ var phrasesEO = map[string]string{
"edit": "redakti",
"move to...": "movi al...",
"pin": "alpingli",
"published with write.as": "Eldonita per write.as",
"share modal ending": "Sendu ĝin al unu amiko, kunhavu ĝin tra la retejo, aŭ eble tweet ĝin. Lernu pli.",
"share modal introduction": "Ĉiu eldonita afiŝo havas sekretan, unikan URL-on, kiun vi povas kunhavigi kun iu ajn. Ĉi tiu estas tiu URL-o:",
"share modal title": "Kunhavigi ĉi tiun afiŝon",
"share": "kunhavigi",
"unpin": "malalpingli",
"published with write.as": "Eldonita per write.as",
"share modal ending": "Sendu ĝin al unu amiko, kunhavu ĝin tra la retejo, aŭ eble tweet ĝin. Lernu pli.",
"share modal introduction": "Ĉiu eldonita afiŝo havas sekretan, unikan URL-on, kiun vi povas kunhavigi kun iu ajn. Ĉi tiu estas tiu URL-o:",
"share modal title": "Kunhavigi ĉi tiun afiŝon",
"share": "kunhavigi",
"unpin": "malalpingli",
}
l10n/phrases_eu.go
+28
View File
@@ -0,0 +1,28 @@
package l10n
var phrasesEU = map[string]string{
"Anonymous post": "Bidalketa anonimoa",
"Blogs": "Blogak",
"Enter": "Sartu",
"Newer": "Berriagoak",
"Older": "Zaharragoak",
"Posts": "Bidalketak",
"Publish to...": "Argitaratu hemen...",
"Publish": "Argitaratu",
"Read more...": "Irakurri gehiago...",
"Subscribe": "Izena eman",
"This blog requires a password.": "Blog honek pasahitza behar du.",
"Toggle theme": "Aldatu itxura",
"View posts": "Ikusi bidalketak",
"delete": "ezabatu",
"edit": "editatu",
"email subscription prompt": "Sartu zure e-posta helbidea berriak jasotzeko.",
"move to...": "mugitu hona...",
"pin": "finkatu",
"published with write.as": "write.as bidez argitaratua",
"share modal ending": "Bidali lagun bati, partekatu sarean zehar edo agian, txiokatu. Ikasi gehiago.",
"share modal introduction": "Argitaratutako bidalketa orok URL sekretu eta bakarra dauka, edonorekin partekatu dezakezuna. Hau da URLa:",
"share modal title": "Partekatu bidalketa hau",
"share": "partekatu",
"unpin": "desfinkatu",
}
l10n/phrases_fr.go
+24 -20
View File
@@ -1,24 +1,28 @@
package l10n
var phrasesFR = map[string]string{
"Anonymous post": "Post anonyme",
"Blogs": "Blogs",
"Newer": "Nouveaux",
"Older": "Anciens",
"Posts": "Posts",
"Publish to...": "Publie sur...",
"Publish": "Publie",
"Read more...": "En savoir plus...",
"Toggle theme": "Activer thème",
"View posts": "Voir Posts",
"delete": "effacer",
"edit": "modifier",
"move to...": "déplacer vers...",
"pin": "épingler",
"published with write.as": "publié avec write.as",
"share modal ending": "Envoie-le à un ami, partage-le sur le web, ou peut-être en tant que tweet. En savoir plus.",
"share modal introduction": "Tout post publié a une adresse URL secrète et unique que tu peux partager avec qui tu veux. Voici cette URL:",
"share modal title": "Partage ce post",
"share": "partager",
"unpin": "enlever",
"Anonymous post": "Billet anonyme",
"Blogs": "Blogs",
"Enter": "Valider",
"Newer": "Récents",
"Older": "Précédents",
"Posts": "Billets",
"Publish to...": "Publier sur...",
"Publish": "Publier",
"Read more...": "Lire la suite...",
"Subscribe": "S'abonner",
"This blog requires a password.": "Ce blog requiert un mot de passe.",
"Toggle theme": "Changer de thème",
"View posts": "Voir les billets",
"delete": "effacer",
"edit": "modifier",
"email subscription prompt": "Insérer votre adresse email pour recevoir les mises à jour",
"move to...": "déplacer vers...",
"pin": "épingler",
"published with write.as": "publié avec write.as",
"share modal ending": "Envoyer à un ami, partager sur le web, ou peut-être en tant que tweet. En savoir plus.",
"share modal introduction": "Chaque billet dispose d’une adresse (URL) secrète et unique qui peut être partagée avec quelqu’un. Voici cette URL:",
"share modal title": "Partager ce billet",
"share": "partager",
"unpin": "détacher",
}
l10n/phrases_gl.go
+28
View File
@@ -0,0 +1,28 @@
package l10n
var phrasesGL = map[string]string{
"Anonymous post": "Publicación anónima",
"Blogs": "Blogs",
"Enter": "Entrar",
"Newer": "Máis recente",
"Older": "Máis antigo",
"Posts": "Publicacións",
"Publish to...": "Publicar en...",
"Publish": "Publicar",
"Read more...": "Saber máis...",
"Subscribe": "Subscribir",
"This blog requires a password.": "Este blog require un contrasinal.",
"Toggle theme": "Cambiar decorado",
"View posts": "Ver Publicacións",
"delete": "eliminar",
"edit": "editar",
"email subscription prompt": "Escribe o teu email para recibir actualizacións.",
"move to...": "mover a...",
"pin": "fixar",
"published with write.as": "publicado con write.as",
"share modal ending": "Envíallo a un amigo, compárteo en internet e redes sociais. Saber máis.",
"share modal introduction": "Cada publicación ten un identificador, un URL único que podes compartir con calquera. Este é o URL:",
"share modal title": "Comparte a publicación",
"share": "compartir",
"unpin": "desafixar",
}
l10n/phrases_he.go
+28
View File
@@ -0,0 +1,28 @@
package l10n
var phrasesHE = map[string]string{
"Anonymous post": "פוסטים אנונימיים",
"Blogs": "בלוגים",
"Enter": "היכנס",
"Newer": "חדש יותר",
"Older": "ישן יותר",
"Posts": "פוסטים",
"Publish to...": "פירסום ל...",
"Publish": "פירסום",
"Read more...": "קרא עוד...",
"Subscribe": "הירשם",
"This blog requires a password.": "לבלוג זה ישנה סיסמה",
"Toggle theme": "החלפת ערכת נושא",
"View posts": "צפייה בפוסטים",
"delete": "מחיקה",
"edit": "עריכה",
"email subscription prompt": "הכנס את המייל שלך כדי לקבל לעידכונים",
"move to...": "העברה ל...",
"pin": "הצמדה",
"published with write.as": "פורסם ע''י write.as",
"share modal ending": "שלח לחברים, פרסם ברשת, או אולי צייץ את זה. למד עוד.",
"share modal introduction": "לכל פוסט יש קישור סודי ויחודי שניתן לשתף עם כל אחד. הנה הקישור:",
"share modal title": "שיתוף פוסט זה",
"share": "שיתוף",
"unpin": "ביטול הצמדה",
}
l10n/phrases_it.go
+24 -20
View File
@@ -1,24 +1,28 @@
package l10n
var phrasesIT = map[string]string{
"Anonymous post": "Post anonimo",
"Blogs": "Blogs",
"Newer": "Più recenti",
"Older": "Più vecchi",
"Posts": "Posts",
"Publish to...": "Pubblica su...",
"Publish": "Pubblica",
"Read more...": "Continua...",
"Toggle theme": "Attiva tema",
"View posts": "Vedi Posts",
"delete": "cancella",
"edit": "modifica",
"move to...": "sposta verso...",
"pin": "attacca",
"published with write.as": "pubblicato con write.as",
"share modal ending": "Mandalo ad un amico, condividilo sul web, oppure tweettalo. Per saperne di più.",
"share modal introduction": "Ogni post pubblicato possiede un URL segreto e unico che puoi condividere con chiunque. Questo è l'URL:",
"share modal title": "Condividi questo post",
"share": "condividi",
"unpin": "stacca",
"Anonymous post": "Post anonimo",
"Blogs": "Blogs",
"Enter": "Invio",
"Newer": "Recenti",
"Older": "Precedenti",
"Posts": "Posts",
"Publish to...": "Pubblica su...",
"Publish": "Pubblica",
"Read more...": "Continua...",
"Subscribe": "Sottoscrivi",
"This blog requires a password.": "Questo blog necessita una password.",
"Toggle theme": "Attiva tema",
"View posts": "Vedi Posts",
"delete": "cancella",
"edit": "modifica",
"email subscription prompt": "Inserisci la tua email per ricevere aggiornamenti",
"move to...": "sposta verso...",
"pin": "appunta",
"published with write.as": "pubblicato con write.as",
"share modal ending": "Mandala ad un amico, condividila sul web, oppure tweettala. Per saperne di più.",
"share modal introduction": "Ogni post pubblicato possiede una URL segreta e unica che puoi condividere con chiunque. Questa è l'URL:",
"share modal title": "Condividi questo post",
"share": "condividi",
"unpin": "stacca",
}
l10n/phrases_ja.go
+24 -20
View File
@@ -1,24 +1,28 @@
package l10n
var phrasesJA = map[string]string{
"Anonymous post": "匿名投稿",
"Blogs": "ブログ",
"Newer": "新しい投稿",
"Older": "古い投稿",
"Posts": "投稿",
"Publish to...": "公開先…",
"Publish": "公開",
"Read more...": "もっと読む…",
"Toggle theme": "テーマを変更",
"View posts": "投稿を見る",
"delete": "削除",
"edit": "編集",
"move to...": "移動…",
"pin": "固定表示する",
"published with write.as": "write.as を使って公開されました",
"share modal ending": "友達に送信したり、ウェブの大海にシェアしたり、ツイートしたり。もっと知る。",
"share modal introduction": "全ての投稿には秘密の、シェアするとだれでも見ることのできる固有のURLがあります。これがそのURLです:",
"share modal title": "投稿をシェアする",
"share": "シェア",
"unpin": "固定表示をやめる",
"Anonymous post": "匿名投稿",
"Blogs": "ブログ",
"Enter": "認証",
"Newer": "新しい投稿",
"Older": "古い投稿",
"Posts": "投稿",
"Publish to...": "公開先…",
"Publish": "公開",
"Read more...": "もっと読む…",
"Subscribe": "購読",
"This blog requires a password.": "このブログはパスワードを必要としています。",
"Toggle theme": "テーマを変更",
"View posts": "投稿を見る",
"delete": "削除",
"edit": "編集",
"email subscription prompt": "このブログを購読したい場合は、メールアドレスを入力してください。",
"move to...": "移動…",
"pin": "固定表示する",
"published with write.as": "write.as を使って公開されました",
"share modal ending": "友達に送信したり、Web 上で共有したり、ツイートすることが出来ます。もっと詳しく。",
"share modal introduction": "全ての投稿には秘密の、シェアするとだれでも見ることのできる固有の URL があります。これがその URL です:",
"share modal title": "投稿をシェアする",
"share": "シェア",
"unpin": "固定表示をやめる",
}
l10n/phrases_nl.go
+28
View File
@@ -0,0 +1,28 @@
package l10n
var phrasesNL = map[string]string{
"Anonymous post": "Anoniem artikel",
"Blogs": "Blogs",
"Enter": "Invoeren",
"Newer": "Nieuwer",
"Older": "Ouder",
"Posts": "Artikelen",
"Publish to...": "Publiceren op...",
"Publish": "Publiceren",
"Read more...": "Lees verder...",
"Subscribe": "Abonneren",
"This blog requires a password.": "Voor dit blog is een wachtwoord vereist.",
"Toggle theme": "Ander thema",
"View posts": "Artikelen bekijken",
"delete": "verwijderen",
"edit": "bewerken",
"email subscription prompt": "Voer je e-mailadres in om te abonneren op updates.",
"move to...": "verplaatsen naar...",
"pin": "vastmaken",
"published with write.as": "gepubliceerd met write.as",
"share modal ending": "Deel de link met een vriend, op het internet of op Twitter. Meer informatie.",
"share modal introduction": "Elk gepubliceerd artikel bevat een geheime, unieke link die je met iedereen kunt delen. Dit is de link:",
"share modal title": "Deel dit artikel",
"share": "delen",
"unpin": "losmaken",
}
l10n/phrases_zh.go
+12 -10
View File
@@ -3,24 +3,26 @@ package l10n
var phrasesZH = map[string]string{
"Anonymous post": "匿名文章",
"Blogs": "博客",
"Enter": "按下回车",
"Newer": "更新",
"Older": "较旧",
"Enter": "进入",
"Newer": "最近的博客",
"Older": "之前的博客",
"Posts": "文章",
"Publish to...": "发布到...",
"Publish": "发布",
"Read more...": "阅读更多",
"This blog requires a password.": "这篇文章需要密码",
"Subscribe": "订阅",
"This blog requires a password.": "这篇博客需要密码",
"Toggle theme": "更换主题",
"View posts": "查看文章",
"delete": "删除",
"edit": "编辑",
"email subscription prompt": "输入邮件地址，订阅更新",
"move to...": "移动到",
"pin": "固定文章",
"published with write.as": "通过 write.as 发布",
"share modal ending": "发送给朋友、与世界分享或者发条推。了解更多。",
"share modal introduction": "发布的每篇文章都有一个唯一的私有网址，你可以把它分享给任何人：",
"share modal title": "分享文章",
"share": "分享",
"unpin": "取消固定",
"published with write.as": "用write.as来发布",
"share modal ending": "发送给朋友，或者线上分享，如果可能的话，分享到推特上。了解更多。",
"share modal introduction": "发布的每篇文章都有一个唯一的私有URL，你可以把它分享给任何人。这是URL：",
"share modal title": "分享这篇文章",
"share": "分享",
"unpin": "取消固定",
}
l10n/strings.go
+12
View File
@@ -6,6 +6,10 @@ func Strings(lang string) map[string]string {
switch lang {
case "ar":
return phrasesAR
case "cs":
return phrasesCS
case "da":
return phrasesDA
case "de":
return phrasesDE
case "el":
@@ -14,10 +18,16 @@ func Strings(lang string) map[string]string {
return phrasesEO
case "es":
return phrasesES
case "eu":
return phrasesEU
case "fa":
return phrasesFA
case "fr":
return phrasesFR
case "gl":
return phrasesGL
case "he":
return phrasesHE
case "hu":
return phrasesHU
case "it":
@@ -28,6 +38,8 @@ func Strings(lang string) map[string]string {
return phrasesLT
case "mk":
return phrasesMK
case "nl":
return phrasesNL
case "pl":
return phrasesPL
case "pt":
posts/parse.go
+189
View File
@@ -1,7 +1,25 @@
package posts
import (
"fmt"
stripmd "github.com/writeas/go-strip-markdown/v2"
"github.com/writeas/slug"
"github.com/writeas/web-core/stringmanip"
"regexp"
"strings"
"unicode"
"unicode/utf8"
)
const (
maxTitleLen = 80
assumedTitleLen = 80
)
var (
titleElementReg = regexp.MustCompile("</?p>")
urlReg = regexp.MustCompile("https?://")
imgReg = regexp.MustCompile(`!\[([^]]+)\]\([^)]+\)`)
)
// ExtractTitle takes the given raw post text and returns a title, if explicitly
@@ -19,3 +37,174 @@ func ExtractTitle(content string) (title string, body string) {
body = content
return
}
func PostTitle(content, friendlyId string) string {
content = StripHTMLWithoutEscaping(content)
content = strings.TrimLeftFunc(stripmd.Strip(content), unicode.IsSpace)
eol := strings.IndexRune(content, '\n')
blankLine := strings.Index(content, "\n\n")
if blankLine != -1 && blankLine <= eol && blankLine <= assumedTitleLen {
return strings.TrimSpace(content[:blankLine])
} else if utf8.RuneCountInString(content) <= maxTitleLen {
return content
}
return friendlyId
}
func FriendlyPostTitle(content, friendlyId string) string {
content = StripHTMLWithoutEscaping(content)
content = strings.TrimLeftFunc(stripmd.Strip(content), unicode.IsSpace)
eol := strings.IndexRune(content, '\n')
blankLine := strings.Index(content, "\n\n")
if blankLine != -1 && blankLine <= eol && blankLine <= assumedTitleLen {
return strings.TrimSpace(content[:blankLine])
} else if eol == -1 && utf8.RuneCountInString(content) <= maxTitleLen {
return content
}
title, truncd := TruncToWord(PostLede(content, true), maxTitleLen)
if truncd {
title += "..."
}
return title
}
// PostDescription generates a description based on the given post content,
// title, and post ID. This doesn't consider a V2 post field, `title` when
// choosing what to generate. In case a post has a title, this function will
// fail, and logic should instead be implemented to skip this when there's no
// title, like so:
//
// var desc string
// if title == "" {
// desc = PostDescription(content, title, friendlyId)
// } else {
// desc = ShortPostDescription(content)
// }
func PostDescription(content, title, friendlyId string) string {
maxLen := 140
if content == "" {
content = "WriteFreely is a painless, simple, federated blogging platform."
} else {
fmtStr := "%s"
truncation := 0
if utf8.RuneCountInString(content) > maxLen {
// Post is longer than the max description, so let's show a better description
fmtStr = "%s..."
truncation = 3
}
if title == friendlyId {
// No specific title was found; simply truncate the post, starting at the beginning
content = fmt.Sprintf(fmtStr, strings.Replace(stringmanip.Substring(content, 0, maxLen-truncation), "\n", " ", -1))
} else {
// There was a title, so return a real description
blankLine := strings.Index(content, "\n\n")
if blankLine < 0 {
blankLine = 0
}
truncd := stringmanip.Substring(content, blankLine, blankLine+maxLen-truncation)
contentNoNL := strings.Replace(truncd, "\n", " ", -1)
content = strings.TrimSpace(fmt.Sprintf(fmtStr, contentNoNL))
}
}
return content
}
func ShortPostDescription(content string) string {
maxLen := 140
fmtStr := "%s"
truncation := 0
if utf8.RuneCountInString(content) > maxLen {
// Post is longer than the max description, so let's show a better description
fmtStr = "%s..."
truncation = 3
}
return strings.TrimSpace(fmt.Sprintf(fmtStr, strings.Replace(stringmanip.Substring(content, 0, maxLen-truncation), "\n", " ", -1)))
}
// TruncToWord truncates the given text to the provided limit.
func TruncToWord(s string, l int) (string, bool) {
truncated := false
c := []rune(s)
if len(c) > l {
truncated = true
s = string(c[:l])
spaceIdx := strings.LastIndexByte(s, ' ')
if spaceIdx > -1 {
s = s[:spaceIdx]
}
}
return s, truncated
}
// PostLede attempts to extract the first thought of the given post, generally
// contained within the first line or sentence of text.
func PostLede(t string, includePunc bool) string {
// Adjust where we truncate if we want to include punctuation
iAdj := 0
if includePunc {
iAdj = 1
}
// Find lede within first line of text
nl := strings.IndexRune(t, '\n')
if nl > -1 {
t = t[:nl]
}
// Strip certain HTML tags
t = titleElementReg.ReplaceAllString(t, "")
// Strip URL protocols
t = urlReg.ReplaceAllString(t, "")
// Strip image URL, leaving only alt text
t = imgReg.ReplaceAllString(t, " $1 ")
// Find lede within first sentence
punc := strings.Index(t, ". ")
if punc > -1 {
t = t[:punc+iAdj]
}
punc = stringmanip.IndexRune(t, '。')
if punc > -1 {
c := []rune(t)
t = string(c[:punc+iAdj])
}
return t
}
func GetSlug(title, lang string) string {
return GetSlugFromPost("", title, lang)
}
func GetSlugFromPost(title, body, lang string) string {
if title == "" {
// Remove Markdown, so e.g. link URLs and image alt text don't make it into the slug
body = strings.TrimSpace(stripmd.StripOptions(body, stripmd.Options{SkipImages: true}))
title = PostTitle(body, body)
}
title = PostLede(title, false)
// Truncate lede if needed
title, _ = TruncToWord(title, maxTitleLen)
var s string
if lang != "" && len(lang) == 2 {
s = slug.MakeLang(title, lang)
} else {
s = slug.Make(title)
}
// Transliteration may cause the slug to expand past the limit, so truncate again
s, _ = TruncToWord(s, maxTitleLen)
return strings.TrimFunc(s, func(r rune) bool {
// TruncToWord doesn't respect words in a slug, since spaces are replaced
// with hyphens. So remove any trailing hyphens.
return r == '-'
})
}
posts/parse_test.go
+42
View File
@@ -33,3 +33,45 @@ Yep.`},
}
}
}
func TestPostLede(t *testing.T) {
text := map[string]string{
"早安。跨出舒適圈，才能前往": "早安。",
"早安。This is my post. It is great.": "早安。",
"Hello. 早安。": "Hello.",
"Sup? Everyone says punctuation is punctuation.": "Sup?",
"Humans are humans, and society is full of good and bad actors. Technology, at the most fundamental level, is a neutral tool that can be used by either to meet any ends. ": "Humans are humans, and society is full of good and bad actors.",
`Online Domino Is Must For Everyone
Do you want to understand how to play poker online?`: "Online Domino Is Must For Everyone",
`おはようございます
私は日本から帰ったばかりです。`: "おはようございます",
"Hello, we say, おはよう. We say \"good morning\"": "Hello, we say, おはよう.",
}
c := 1
for i, o := range text {
if s := PostLede(i, true); s != o {
t.Errorf("#%d: Got '%s' from '%s'; expected '%s'", c, s, i, o)
}
c++
}
}
func TestTruncToWord(t *testing.T) {
text := map[string]string{
"Можливо, ми можемо використовувати інтернет-інструменти, щоб виготовити якийсь текст, який би міг бути і на, і в кінцевому підсумку, буде скорочено, тому що це тривало так довго.": "Можливо, ми можемо використовувати інтернет-інструменти, щоб виготовити якийсь",
"早安。This is my post. It is great. It is a long post that is great that is a post that is great.": "早安。This is my post. It is great. It is a long post that is great that is a post",
"Sup? Everyone says punctuation is punctuation.": "Sup? Everyone says punctuation is punctuation.",
"I arrived in Japan six days ago. Tired from a 10-hour flight after a night-long layover in Calgary, I wandered wide-eyed around Narita airport looking for an ATM.": "I arrived in Japan six days ago. Tired from a 10-hour flight after a night-long",
}
c := 1
for i, o := range text {
if s, _ := TruncToWord(i, 80); s != o {
t.Errorf("#%d: Got '%s' from '%s'; expected '%s'", c, s, i, o)
}
c++
}
}
posts/render.go
+35
View File
@@ -3,6 +3,7 @@ package posts
import (
"github.com/microcosm-cc/bluemonday"
"github.com/writeas/saturday"
"html"
"regexp"
"strings"
"unicode"
@@ -61,3 +62,37 @@ func ApplyBasicMarkdown(data []byte) string {
return outHTML
}
// ApplyBasicAccessibleMarkdown applies Markdown to the given data, rendering basic text formatting and preserving hard
// line breaks in HTML. It is meant for formatting text in small, multi-line UI elements, like user profile biographies.
func ApplyBasicAccessibleMarkdown(data []byte) string {
mdExtensions := 0 |
blackfriday.EXTENSION_STRIKETHROUGH |
blackfriday.EXTENSION_SPACE_HEADERS |
blackfriday.EXTENSION_HEADER_IDS |
blackfriday.EXTENSION_HARD_LINE_BREAK
htmlFlags := 0 |
blackfriday.HTML_USE_SMARTYPANTS |
blackfriday.HTML_USE_XHTML |
blackfriday.HTML_SMARTYPANTS_DASHES
// Generate Markdown
md := blackfriday.Markdown([]byte(data), blackfriday.HtmlRenderer(htmlFlags, "", ""), mdExtensions)
// Strip out bad HTML
policy := bluemonday.UGCPolicy()
policy.AllowAttrs("class", "id").Globally()
policy.AllowAttrs("rel").OnElements("a")
policy.RequireNoFollowOnLinks(false)
outHTML := string(policy.SanitizeBytes(md))
// Strip surrounding <p> tags that blackfriday adds
outHTML = markeddownReg.ReplaceAllString(outHTML, "$1")
outHTML = strings.TrimRightFunc(outHTML, unicode.IsSpace)
return outHTML
}
// StripHTMLWithoutEscaping strips HTML tags with bluemonday's StrictPolicy, then unescapes the HTML
// entities added in by sanitizing the content.
func StripHTMLWithoutEscaping(content string) string {
return html.UnescapeString(bluemonday.StrictPolicy().Sanitize(content))
}