Mercurial > silorider

annotate tests/test_silos_twitter.py @ 60:b7da3d97ea99

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add profile URL handlers Silos register these handlers so that everybody knows if a hyperlink is a mention to another user on a particular social network. If any handler matches, silos not related to that social media will skip that link. It's possible than in rare cases we want that link everywhere, but so far I haven't needed it, compared to all the times I didn't want these links.
author Ludovic Chabant <ludovic@chabant.com>
date Sat, 28 Oct 2023 11:57:04 -0700
parents d65f6dced79f
children dafbbf25bfc8
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import pytest
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
2 from .mockutil import mock_urllib
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 def test_one_article(cli, feedutil, tweetmock):
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6 feed = cli.createTempFeed(feedutil.makeFeed(
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7 """<h1 class="p-name">A new article</h1>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8 <div class="e-content">
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9 <p>This is the text of the article.</p>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10 <p>It has 2 paragraphs.</p>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11 </div>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12 <a class="u-url" href="https://example.org/a-new-article">permalink</a>"""
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13 ))
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15 cli.appendSiloConfig('test', 'twitter', url='/blah')
18
a921cc2306bc Do our own HTML parsing/stripping of micropost contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 4
diff changeset
16 cli.setFeedConfig('feed', feed)
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17 tweetmock.installTokens(cli, 'test')
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18
18
a921cc2306bc Do our own HTML parsing/stripping of micropost contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 4
diff changeset
19 ctx, _ = cli.run('process')
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
20 assert ctx.cache.wasPosted('test', 'https://example.org/a-new-article')
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
21 toot = ctx.silos[0].client.tweets[0]
4
c199bd681e4e Twitter API accepts direct URLs for media.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
22 assert toot == ('A new article https://example.org/a-new-article', [])
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
25 def test_one_micropost(cli, feedutil, tweetmock):
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
26 feed = cli.createTempFeed(feedutil.makeFeed(
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
27 """<p class="p-name">This is a quick update.</p>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
28 <a class="u-url" href="/01234.html">permalink</a>"""
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
29 ))
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
30
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
31 cli.appendSiloConfig('test', 'twitter', url='/blah')
18
a921cc2306bc Do our own HTML parsing/stripping of micropost contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 4
diff changeset
32 cli.setFeedConfig('feed', feed)
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
33 tweetmock.installTokens(cli, 'test')
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
34
18
a921cc2306bc Do our own HTML parsing/stripping of micropost contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 4
diff changeset
35 ctx, _ = cli.run('process')
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
36 assert ctx.cache.wasPosted('test', '/01234.html')
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
37 toot = ctx.silos[0].client.tweets[0]
4
c199bd681e4e Twitter API accepts direct URLs for media.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
38 assert toot == ("This is a quick update.", [])
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
39
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
40
25
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
41 def test_one_micropost_with_mention(cli, feedutil, tweetmock):
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
42 feed = cli.createTempFeed(feedutil.makeFeed(
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
43 """<p class="p-name">Hey <a href="https://twitter.com/jack">Jacky</a>
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
44 you should fix your stuff!</p>
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
45 <a class="u-url" href="/01234.html">permalink</a>"""
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
46 ))
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
47
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
48 cli.appendSiloConfig('test', 'twitter', url='/blah')
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
49 cli.setFeedConfig('feed', feed)
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
50 tweetmock.installTokens(cli, 'test')
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
51
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
52 ctx, _ = cli.run('process')
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
53 assert ctx.cache.wasPosted('test', '/01234.html')
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
54 toot = ctx.silos[0].client.tweets[0]
33
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
55 assert toot == ("Hey @jack\nyou should fix your stuff!", [])
25
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
56
fb93d3fbff4e Support transforming twitter profile URLs into mentions.
Ludovic Chabant <ludovic@chabant.com>
parents: 18
diff changeset
57
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
58 def test_one_micropost_with_one_photo(cli, feedutil, tweetmock, monkeypatch):
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
59 feed = cli.createTempFeed(feedutil.makeFeed(
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
60 """<p class="p-name">This is a quick photo update.</p>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
61 <div>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
62 <a class="u-photo" href="/fullimg.jpg"><img src="/thumbimg.jpg"/></a>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
63 </div>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
64 <a class="u-url" href="/01234.html">permalink</a>"""
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
65 ))
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
66
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
67 cli.appendSiloConfig('test', 'twitter', url='/blah')
18
a921cc2306bc Do our own HTML parsing/stripping of micropost contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 4
diff changeset
68 cli.setFeedConfig('feed', feed)
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
69 tweetmock.installTokens(cli, 'test')
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
70
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
71 with monkeypatch.context() as m:
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
72 import silorider.silos.twitter
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
73 mock_urllib(m)
58
d65f6dced79f Fix media callback patches in unit tests
Ludovic Chabant <ludovic@chabant.com>
parents: 39
diff changeset
74 m.setattr(silorider.silos.twitter.TwitterSilo, 'mediaCallback',
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
75 _patched_media_callback)
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
76 ctx, _ = cli.run('process')
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
77
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
78 assert ctx.cache.wasPosted('test', '/01234.html')
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
79 toot = ctx.silos[0].client.tweets[0]
4
c199bd681e4e Twitter API accepts direct URLs for media.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
80 assert toot == ("This is a quick photo update.", ['/fullimg.jpg'])
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
81
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
82
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
83 def test_one_micropost_with_two_photos(cli, feedutil, tweetmock, monkeypatch):
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
84 feed = cli.createTempFeed(feedutil.makeFeed(
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
85 """<p class="p-name">This is a photo update with 2 photos.</p>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
86 <div>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
87 <a class="u-photo" href="/fullimg1.jpg"><img src="/thumbimg1.jpg"/></a>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
88 <a class="u-photo" href="/fullimg2.jpg"><img src="/thumbimg2.jpg"/></a>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
89 </div>
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
90 <a class="u-url" href="/01234.html">permalink</a>"""
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
91 ))
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
92
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
93 cli.appendSiloConfig('test', 'twitter', url='/blah')
18
a921cc2306bc Do our own HTML parsing/stripping of micropost contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 4
diff changeset
94 cli.setFeedConfig('feed', feed)
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
95 tweetmock.installTokens(cli, 'test')
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
96
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
97 with monkeypatch.context() as m:
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
98 import silorider.silos.twitter
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
99 mock_urllib(m)
58
d65f6dced79f Fix media callback patches in unit tests
Ludovic Chabant <ludovic@chabant.com>
parents: 39
diff changeset
100 m.setattr(silorider.silos.twitter.TwitterSilo, 'mediaCallback',
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
101 _patched_media_callback)
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
102 ctx, _ = cli.run('process')
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
103
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
104 assert ctx.cache.wasPosted('test', '/01234.html')
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
105 toot = ctx.silos[0].client.tweets[0]
4
c199bd681e4e Twitter API accepts direct URLs for media.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
106 assert toot == ("This is a photo update with 2 photos.",
c199bd681e4e Twitter API accepts direct URLs for media.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
107 ['/fullimg1.jpg', '/fullimg2.jpg'])
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
108
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
109
33
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
110 def test_micropost_with_long_text_and_link(cli, feedutil, tweetmock, monkeypatch):
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
111 feed = cli.createTempFeed(feedutil.makeFeed(
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
112 """<div class="p-name">
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
113 <p>This a pretty long text that has a link in it :) We want to make sure it gets to the limit of what Twitter allows, so that we can test there won't be any off-by-one errors in measurements. Here is a <a href="https://docs.python.org/3/library/textwrap.html">link to Python's textwrap module</a>, which is appropriate!!!</p>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
114 </div>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
115 <a class="u-url" href="/01234.html">permalink</a>"""
33
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
116 ))
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
117
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
118 cli.appendSiloConfig('test', 'twitter', url='/blah')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
119 cli.setFeedConfig('feed', feed)
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
120 tweetmock.installTokens(cli, 'test')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
121
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
122 ctx, _ = cli.run('process')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
123 assert ctx.cache.wasPosted('test', '/01234.html')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
124 toot = ctx.silos[0].client.tweets[0]
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
125 assert toot == ("This a pretty long text that has a link in it :) We want to make sure it gets to the limit of what Twitter allows, so that we can test there won't be any off-by-one errors in measurements. Here is a link to Python's textwrap module, which is appropriate!!! https://docs.python.org/3/library/textwrap.html",
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
126 [])
33
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
127
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
128
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
129 def test_micropost_with_too_long_text_and_link_1(cli, feedutil, tweetmock, monkeypatch):
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
130 feed = cli.createTempFeed(feedutil.makeFeed(
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
131 """<div class="p-name">
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
132 <p>This time we have a text that's slightly too long, with <a href="https://thisdoesntmatter.com">a link here</a>. We'll be one character too long, with a short word at the end to test the shortening algorithm. Otherwise, don't worry about it. Blah blah blah. Trying to get to the limit. Almost here yes</p>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
133 </div>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
134 <a class="u-url" href="/01234.html">permalink</a>"""
33
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
135 ))
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
136
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
137 cli.appendSiloConfig('test', 'twitter', url='/blah')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
138 cli.setFeedConfig('feed', feed)
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
139 tweetmock.installTokens(cli, 'test')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
140
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
141 ctx, _ = cli.run('process')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
142 assert ctx.cache.wasPosted('test', '/01234.html')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
143 toot = ctx.silos[0].client.tweets[0]
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
144 assert toot == ("This time we have a text that's slightly too long, with a link here. We'll be one character too long, with a short word at the end to test the shortening algorithm. Otherwise, don't worry about it. Blah blah blah. Trying to get to the limit. Almost here... /01234.html",
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
145 [])
33
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
146
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
147
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
148 def test_micropost_with_too_long_text_and_link_2(cli, feedutil, tweetmock, monkeypatch):
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
149 feed = cli.createTempFeed(feedutil.makeFeed(
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
150 """<div class="p-name">
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
151 <p>This time we have a text that's slightly too long, with <a href="https://thisdoesntmatter.com">a link here</a>. We'll be one character too long, with a loooooong word at the end to test the shortening algorithm. Otherwise, don't worry about it. Blah blah blah. Our long word is: califragilisticastuff</p>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
152 </div>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
153 <a class="u-url" href="/01234.html">permalink</a>"""
33
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
154 ))
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
155
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
156 cli.appendSiloConfig('test', 'twitter', url='/blah')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
157 cli.setFeedConfig('feed', feed)
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
158 tweetmock.installTokens(cli, 'test')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
159
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
160 ctx, _ = cli.run('process')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
161 assert ctx.cache.wasPosted('test', '/01234.html')
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
162 toot = ctx.silos[0].client.tweets[0]
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
163 assert toot == ("This time we have a text that's slightly too long, with a link here. We'll be one character too long, with a loooooong word at the end to test the shortening algorithm. Otherwise, don't worry about it. Blah blah blah. Our long word is:... /01234.html",
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
164 [])
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
165
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
166
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
167 def _patched_media_callback(self, tmpfile, mt, url, desc):
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
168 return self.client.simple_upload(url)
33
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
169
9e4eb3f2754e Improve handling of character limits in html stripping
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
170
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
171 @pytest.fixture(scope='session')
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
172 def tweetmock():
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
173 from silorider.silos.twitter import TwitterSilo
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
174 TwitterSilo._CLIENT_CLASS = TwitterMock
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
175 return TwitterMockUtil()
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
176
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
177
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
178 class TwitterMock:
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
179 def __init__(self, consumer_key, consumer_secret,
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
180 access_token_key, access_token_secret):
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
181 assert consumer_key == 'TEST_CLIENT_KEY'
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
182 assert consumer_secret == 'TEST_CLIENT_SECRET'
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
183 assert access_token_key == 'TEST_ACCESS_KEY'
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
184 assert access_token_secret == 'TEST_ACCESS_SECRET'
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
185
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
186 self.tweets = []
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
187 self.media = []
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
188
39
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
189 def create_tweet(self, text, media_ids=None):
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
190 media_names = []
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
191 if media_ids:
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
192 for mid in media_ids:
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
193 assert(self.media[mid] is not None)
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
194 media_names.append(self.media[mid])
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
195 self.media[mid] = None
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
196 assert all([m is None for m in self.media])
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
197
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
198 self.tweets.append((text, media_names))
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
199 self.media = []
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
200
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
201 def simple_upload(self, fname, file=None):
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
202 self.media.append(fname)
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API
Ludovic Chabant <ludovic@chabant.com>
parents: 33
diff changeset
203 return len(self.media) - 1
2
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
204
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
205
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
206 class TwitterMockUtil:
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
207 def installTokens(self, cli, silo_name):
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
208 def do_install_tokens(ctx):
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
209 ctx.cache.setCustomValue(
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
210 '%s_clienttoken' % silo_name,
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
211 'TEST_CLIENT_KEY,TEST_CLIENT_SECRET')
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
212 ctx.cache.setCustomValue(
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
213 '%s_accesstoken' % silo_name,
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
214 'TEST_ACCESS_KEY,TEST_ACCESS_SECRET')
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
215
27543b2e73b9 Add Twitter silo.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
216 cli.preExecHook(do_install_tokens)