silorider: tests/test_silos_twitter.py annotate

annotate tests/test_silos_twitter.py @ 78:932aa9922d98

Add hashtag facets to Bluesky silo

author	Ludovic Chabant <ludovic@chabant.com>
date	Wed, 28 Aug 2024 09:38:33 -0700
parents	dafbbf25bfc8
children

rev	line source
69 dafbbf25bfc8 Upgrade bluesky silo to atproto 0.0.35 and unit tests. Ludovic Chabant <ludovic@chabant.com> parents: 58 diff changeset	1 import os.path
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import pytest
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	3 from .mockutil import mock_urllib
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	6 def test_one_article(cli, feedutil, tweetmock):
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	7 feed = cli.createTempFeed(feedutil.makeFeed(
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	8 """<h1 class="p-name">A new article</h1>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	9 <div class="e-content">
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	10 <p>This is the text of the article.</p>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	11 <p>It has 2 paragraphs.</p>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12 </div>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	13 <a class="u-url" href="https://example.org/a-new-article">permalink</a>"""
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14 ))
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	15
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	16 cli.appendSiloConfig('test', 'twitter', url='/blah')
18 a921cc2306bc Do our own HTML parsing/stripping of micropost contents. Ludovic Chabant <ludovic@chabant.com> parents: 4 diff changeset	17 cli.setFeedConfig('feed', feed)
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18 tweetmock.installTokens(cli, 'test')
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19
18 a921cc2306bc Do our own HTML parsing/stripping of micropost contents. Ludovic Chabant <ludovic@chabant.com> parents: 4 diff changeset	20 ctx, _ = cli.run('process')
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21 assert ctx.cache.wasPosted('test', 'https://example.org/a-new-article')
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	22 toot = ctx.silos[0].client.tweets[0]
4 c199bd681e4e Twitter API accepts direct URLs for media. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	23 assert toot == ('A new article https://example.org/a-new-article', [])
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 def test_one_micropost(cli, feedutil, tweetmock):
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 feed = cli.createTempFeed(feedutil.makeFeed(
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	28 """<p class="p-name">This is a quick update.</p>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	29 <a class="u-url" href="/01234.html">permalink</a>"""
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	30 ))
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	31
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	32 cli.appendSiloConfig('test', 'twitter', url='/blah')
18 a921cc2306bc Do our own HTML parsing/stripping of micropost contents. Ludovic Chabant <ludovic@chabant.com> parents: 4 diff changeset	33 cli.setFeedConfig('feed', feed)
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	34 tweetmock.installTokens(cli, 'test')
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	35
18 a921cc2306bc Do our own HTML parsing/stripping of micropost contents. Ludovic Chabant <ludovic@chabant.com> parents: 4 diff changeset	36 ctx, _ = cli.run('process')
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	37 assert ctx.cache.wasPosted('test', '/01234.html')
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	38 toot = ctx.silos[0].client.tweets[0]
4 c199bd681e4e Twitter API accepts direct URLs for media. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	39 assert toot == ("This is a quick update.", [])
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	40
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	41
25 fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	42 def test_one_micropost_with_mention(cli, feedutil, tweetmock):
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	43 feed = cli.createTempFeed(feedutil.makeFeed(
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	44 """<p class="p-name">Hey <a href="https://twitter.com/jack">Jacky</a>
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	45 you should fix your stuff!</p>
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	46 <a class="u-url" href="/01234.html">permalink</a>"""
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	47 ))
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	48
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	49 cli.appendSiloConfig('test', 'twitter', url='/blah')
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	50 cli.setFeedConfig('feed', feed)
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	51 tweetmock.installTokens(cli, 'test')
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	52
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	53 ctx, _ = cli.run('process')
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	54 assert ctx.cache.wasPosted('test', '/01234.html')
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	55 toot = ctx.silos[0].client.tweets[0]
33 9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	56 assert toot == ("Hey @jack\nyou should fix your stuff!", [])
25 fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	57
fb93d3fbff4e Support transforming twitter profile URLs into mentions. Ludovic Chabant <ludovic@chabant.com> parents: 18 diff changeset	58
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	59 def test_one_micropost_with_one_photo(cli, feedutil, tweetmock, monkeypatch):
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	60 feed = cli.createTempFeed(feedutil.makeFeed(
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	61 """<p class="p-name">This is a quick photo update.</p>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	62 <div>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	63 <a class="u-photo" href="/fullimg.jpg"><img src="/thumbimg.jpg"/></a>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	64 </div>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	65 <a class="u-url" href="/01234.html">permalink</a>"""
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	66 ))
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	67
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	68 cli.appendSiloConfig('test', 'twitter', url='/blah')
18 a921cc2306bc Do our own HTML parsing/stripping of micropost contents. Ludovic Chabant <ludovic@chabant.com> parents: 4 diff changeset	69 cli.setFeedConfig('feed', feed)
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	70 tweetmock.installTokens(cli, 'test')
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	71
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	72 with monkeypatch.context() as m:
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	73 import silorider.silos.twitter
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	74 mock_urllib(m)
69 dafbbf25bfc8 Upgrade bluesky silo to atproto 0.0.35 and unit tests. Ludovic Chabant <ludovic@chabant.com> parents: 58 diff changeset	75 m.setattr(os.path, 'getsize', lambda path: 42)
58 d65f6dced79f Fix media callback patches in unit tests Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	76 m.setattr(silorider.silos.twitter.TwitterSilo, 'mediaCallback',
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	77 _patched_media_callback)
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	78 ctx, _ = cli.run('process')
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 assert ctx.cache.wasPosted('test', '/01234.html')
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81 toot = ctx.silos[0].client.tweets[0]
4 c199bd681e4e Twitter API accepts direct URLs for media. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	82 assert toot == ("This is a quick photo update.", ['/fullimg.jpg'])
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	83
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	84
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	85 def test_one_micropost_with_two_photos(cli, feedutil, tweetmock, monkeypatch):
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	86 feed = cli.createTempFeed(feedutil.makeFeed(
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	87 """<p class="p-name">This is a photo update with 2 photos.</p>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88 <div>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	89 <a class="u-photo" href="/fullimg1.jpg"><img src="/thumbimg1.jpg"/></a>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	90 <a class="u-photo" href="/fullimg2.jpg"><img src="/thumbimg2.jpg"/></a>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	91 </div>
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	92 <a class="u-url" href="/01234.html">permalink</a>"""
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	93 ))
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	94
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	95 cli.appendSiloConfig('test', 'twitter', url='/blah')
18 a921cc2306bc Do our own HTML parsing/stripping of micropost contents. Ludovic Chabant <ludovic@chabant.com> parents: 4 diff changeset	96 cli.setFeedConfig('feed', feed)
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	97 tweetmock.installTokens(cli, 'test')
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	98
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	99 with monkeypatch.context() as m:
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	100 import silorider.silos.twitter
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	101 mock_urllib(m)
69 dafbbf25bfc8 Upgrade bluesky silo to atproto 0.0.35 and unit tests. Ludovic Chabant <ludovic@chabant.com> parents: 58 diff changeset	102 m.setattr(os.path, 'getsize', lambda path: 42)
58 d65f6dced79f Fix media callback patches in unit tests Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	103 m.setattr(silorider.silos.twitter.TwitterSilo, 'mediaCallback',
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	104 _patched_media_callback)
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	105 ctx, _ = cli.run('process')
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	106
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	107 assert ctx.cache.wasPosted('test', '/01234.html')
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	108 toot = ctx.silos[0].client.tweets[0]
4 c199bd681e4e Twitter API accepts direct URLs for media. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	109 assert toot == ("This is a photo update with 2 photos.",
c199bd681e4e Twitter API accepts direct URLs for media. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	110 ['/fullimg1.jpg', '/fullimg2.jpg'])
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	111
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	112
33 9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	113 def test_micropost_with_long_text_and_link(cli, feedutil, tweetmock, monkeypatch):
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	114 feed = cli.createTempFeed(feedutil.makeFeed(
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	115 """<div class="p-name">
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	116 <p>This a pretty long text that has a link in it :) We want to make sure it gets to the limit of what Twitter allows, so that we can test there won't be any off-by-one errors in measurements. Here is a <a href="https://docs.python.org/3/library/textwrap.html">link to Python's textwrap module</a>, which is appropriate!!!</p>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	117 </div>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	118 <a class="u-url" href="/01234.html">permalink</a>"""
33 9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	119 ))
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	120
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	121 cli.appendSiloConfig('test', 'twitter', url='/blah')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	122 cli.setFeedConfig('feed', feed)
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	123 tweetmock.installTokens(cli, 'test')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	124
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	125 ctx, _ = cli.run('process')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	126 assert ctx.cache.wasPosted('test', '/01234.html')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	127 toot = ctx.silos[0].client.tweets[0]
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	128 assert toot == ("This a pretty long text that has a link in it :) We want to make sure it gets to the limit of what Twitter allows, so that we can test there won't be any off-by-one errors in measurements. Here is a link to Python's textwrap module, which is appropriate!!! https://docs.python.org/3/library/textwrap.html",
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	129 [])
33 9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	130
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	131
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	132 def test_micropost_with_too_long_text_and_link_1(cli, feedutil, tweetmock, monkeypatch):
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	133 feed = cli.createTempFeed(feedutil.makeFeed(
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	134 """<div class="p-name">
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	135 <p>This time we have a text that's slightly too long, with <a href="https://thisdoesntmatter.com">a link here</a>. We'll be one character too long, with a short word at the end to test the shortening algorithm. Otherwise, don't worry about it. Blah blah blah. Trying to get to the limit. Almost here yes</p>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	136 </div>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	137 <a class="u-url" href="/01234.html">permalink</a>"""
33 9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	138 ))
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	139
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	140 cli.appendSiloConfig('test', 'twitter', url='/blah')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	141 cli.setFeedConfig('feed', feed)
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	142 tweetmock.installTokens(cli, 'test')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	143
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	144 ctx, _ = cli.run('process')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	145 assert ctx.cache.wasPosted('test', '/01234.html')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	146 toot = ctx.silos[0].client.tweets[0]
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	147 assert toot == ("This time we have a text that's slightly too long, with a link here. We'll be one character too long, with a short word at the end to test the shortening algorithm. Otherwise, don't worry about it. Blah blah blah. Trying to get to the limit. Almost here... /01234.html",
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	148 [])
33 9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	149
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	150
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	151 def test_micropost_with_too_long_text_and_link_2(cli, feedutil, tweetmock, monkeypatch):
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	152 feed = cli.createTempFeed(feedutil.makeFeed(
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	153 """<div class="p-name">
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	154 <p>This time we have a text that's slightly too long, with <a href="https://thisdoesntmatter.com">a link here</a>. We'll be one character too long, with a loooooong word at the end to test the shortening algorithm. Otherwise, don't worry about it. Blah blah blah. Our long word is: califragilisticastuff</p>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	155 </div>
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	156 <a class="u-url" href="/01234.html">permalink</a>"""
33 9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	157 ))
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	158
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	159 cli.appendSiloConfig('test', 'twitter', url='/blah')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	160 cli.setFeedConfig('feed', feed)
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	161 tweetmock.installTokens(cli, 'test')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	162
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	163 ctx, _ = cli.run('process')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	164 assert ctx.cache.wasPosted('test', '/01234.html')
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	165 toot = ctx.silos[0].client.tweets[0]
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	166 assert toot == ("This time we have a text that's slightly too long, with a link here. We'll be one character too long, with a loooooong word at the end to test the shortening algorithm. Otherwise, don't worry about it. Blah blah blah. Our long word is:... /01234.html",
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	167 [])
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	168
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	169
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	170 def _patched_media_callback(self, tmpfile, mt, url, desc):
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	171 return self.client.simple_upload(url)
33 9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	172
9e4eb3f2754e Improve handling of character limits in html stripping Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	173
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	174 @pytest.fixture(scope='session')
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	175 def tweetmock():
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	176 from silorider.silos.twitter import TwitterSilo
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	177 TwitterSilo._CLIENT_CLASS = TwitterMock
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	178 return TwitterMockUtil()
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	179
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	180
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	181 class TwitterMock:
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	182 def __init__(self, consumer_key, consumer_secret,
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	183 access_token_key, access_token_secret):
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	184 assert consumer_key == 'TEST_CLIENT_KEY'
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	185 assert consumer_secret == 'TEST_CLIENT_SECRET'
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	186 assert access_token_key == 'TEST_ACCESS_KEY'
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	187 assert access_token_secret == 'TEST_ACCESS_SECRET'
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	188
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	189 self.tweets = []
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	190 self.media = []
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	191
39 c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	192 def create_tweet(self, text, media_ids=None):
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	193 media_names = []
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	194 if media_ids:
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	195 for mid in media_ids:
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	196 assert(self.media[mid] is not None)
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	197 media_names.append(self.media[mid])
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	198 self.media[mid] = None
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	199 assert all([m is None for m in self.media])
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	200
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	201 self.tweets.append((text, media_names))
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	202 self.media = []
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	203
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	204 def simple_upload(self, fname, file=None):
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	205 self.media.append(fname)
c5f73ebb43a5 Replace python-twitter with tweepy to use Twitter's V2 API Ludovic Chabant <ludovic@chabant.com> parents: 33 diff changeset	206 return len(self.media) - 1
2 27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	207
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	208
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	209 class TwitterMockUtil:
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	210 def installTokens(self, cli, silo_name):
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	211 def do_install_tokens(ctx):
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	212 ctx.cache.setCustomValue(
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	213 '%s_clienttoken' % silo_name,
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	214 'TEST_CLIENT_KEY,TEST_CLIENT_SECRET')
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	215 ctx.cache.setCustomValue(
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	216 '%s_accesstoken' % silo_name,
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	217 'TEST_ACCESS_KEY,TEST_ACCESS_SECRET')
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	218
27543b2e73b9 Add Twitter silo. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	219 cli.preExecHook(do_install_tokens)

Mercurial > silorider

annotate tests/test_silos_twitter.py @ 78:932aa9922d98