272 lines
35 KiB
Plaintext
272 lines
35 KiB
Plaintext
.\.venv312\Scripts\python.exe : `torch_dtype` is deprecated! Use `dtype` instead!
|
|
At line:1 char:67
|
|
+ ... df" -Force; .\.venv312\Scripts\python.exe run_pipeline.py --pages 3 > ...
|
|
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
+ CategoryInfo : NotSpecified: (`torch_dtype` i...dtype` instead!:String) [], RemoteException
|
|
+ FullyQualifiedErrorId : NativeCommandError
|
|
|
|
Found file to process: test_full.pdf
|
|
Limiting processing to specific pages: [3]
|
|
Starting full pipeline extraction on input\test_full.pdf
|
|
Output directory: output\20260614_123256
|
|
|
|
Loaded detection model vikp/surya_layout3 on device cpu with dtype torch.float32
|
|
|
|
Detecting bboxes: 0%| | 0/1 [00:00<?, ?it/s]
|
|
Detecting bboxes: 100%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê| 1/1 [00:11<00:00, 11.64s/it]
|
|
Detecting bboxes: 100%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê| 1/1 [00:11<00:00, 11.64s/it]
|
|
|
|
======================================================================
|
|
LAYOUT DETECTION RESULTS: 60 regions found
|
|
======================================================================
|
|
Region types: {'text': 41, 'paragraph_title': 12, 'image': 7}
|
|
|
|
Region 1: type=image bbox=[ 88, 240, 1179, 745] size= 1091x 505px conf=1.00
|
|
Region 2: type=image bbox=[ 3075, 267, 4074, 867] size= 999x 600px conf=1.00
|
|
Region 3: type=image bbox=[ 1110, 284, 3029, 894] size= 1919x 610px conf=1.00
|
|
Region 4: type=image bbox=[ 1866, 1338, 2428, 1887] size= 562x 549px conf=1.00
|
|
Region 5: type=image bbox=[ 582, 2916, 2079, 3815] size= 1497x 899px conf=1.00
|
|
Region 6: type=image bbox=[ 1570, 4147, 2074, 4536] size= 504x 389px conf=1.00
|
|
Region 7: type=image bbox=[ 127, 4652, 3145, 5450] size= 3018x 798px conf=1.00
|
|
Region 8: type=paragraph_title bbox=[ 168, 1018, 1078, 1083] size= 910x 65px conf=1.00
|
|
Region 9: type=paragraph_title bbox=[ 1246, 1019, 1792, 1081] size= 546x 62px conf=1.00
|
|
Region 10: type=paragraph_title bbox=[ 147, 1074, 607, 1133] size= 460x 59px conf=1.00
|
|
Region 11: type=paragraph_title bbox=[ 143, 1758, 1132, 1881] size= 989x 123px conf=1.00
|
|
Region 12: type=paragraph_title bbox=[ 2118, 2592, 4026, 2736] size= 1908x 144px conf=1.00
|
|
Region 13: type=paragraph_title bbox=[ 184, 2593, 2008, 2791] size= 1824x 198px conf=1.00
|
|
Region 14: type=paragraph_title bbox=[ 471, 2787, 496, 2794] size= 25x 7px conf=1.00
|
|
Region 15: type=paragraph_title bbox=[ 2787, 3353, 3387, 3421] size= 600x 68px conf=1.00
|
|
Region 16: type=paragraph_title bbox=[ 132, 6396, 625, 6471] size= 493x 75px conf=1.00
|
|
Region 17: type=paragraph_title bbox=[ 1271, 6399, 1743, 6464] size= 472x 65px conf=1.00
|
|
Region 18: type=paragraph_title bbox=[ 3596, 6398, 4052, 6467] size= 456x 69px conf=1.00
|
|
Region 19: type=paragraph_title bbox=[ 2452, 6401, 2920, 6463] size= 468x 62px conf=1.00
|
|
Region 20: type=text bbox=[ 1829, 103, 2255, 204] size= 426x 101px conf=1.00
|
|
Region 21: type=text bbox=[ 3089, 114, 3784, 181] size= 695x 67px conf=1.00
|
|
Region 22: type=text bbox=[ 198, 122, 759, 183] size= 561x 61px conf=1.00
|
|
Region 23: type=text bbox=[ 237, 738, 1019, 802] size= 782x 64px conf=1.00
|
|
Region 24: type=text bbox=[ 2145, 809, 3121, 1028] size= 976x 219px conf=1.00
|
|
Region 25: type=text bbox=[ 155, 833, 1107, 1035] size= 952x 202px conf=1.00
|
|
Region 26: type=text bbox=[ 3367, 860, 3781, 923] size= 414x 63px conf=1.00
|
|
Region 27: type=text bbox=[ 1252, 944, 1942, 1005] size= 690x 61px conf=1.00
|
|
Region 28: type=text bbox=[ 3080, 946, 4030, 1056] size= 950x 110px conf=1.00
|
|
Region 29: type=text bbox=[ 2149, 1011, 3125, 1922] size= 976x 911px conf=1.00
|
|
Region 30: type=text bbox=[ 1394, 1025, 1866, 1090] size= 472x 65px conf=1.00
|
|
Region 31: type=text bbox=[ 3125, 1062, 3281, 1128] size= 156x 66px conf=1.00
|
|
Region 32: type=text bbox=[ 1240, 1097, 2068, 1173] size= 828x 76px conf=1.00
|
|
Region 33: type=text bbox=[ 3076, 1115, 4072, 1921] size= 996x 806px conf=1.00
|
|
Region 34: type=text bbox=[ 94, 1130, 1173, 1689] size= 1079x 559px conf=1.00
|
|
Region 35: type=text bbox=[ 1234, 1189, 2126, 1259] size= 892x 70px conf=1.00
|
|
Region 36: type=text bbox=[ 1203, 1288, 2272, 2481] size= 1069x 1193px conf=1.00
|
|
Region 37: type=text bbox=[ 140, 1883, 889, 1947] size= 749x 64px conf=1.00
|
|
Region 38: type=text bbox=[ 94, 1934, 1168, 2506] size= 1074x 572px conf=1.00
|
|
Region 39: type=text bbox=[ 2169, 1943, 4048, 2468] size= 1879x 525px conf=1.00
|
|
Region 40: type=text bbox=[ 2108, 2746, 2801, 3550] size= 693x 804px conf=1.00
|
|
Region 41: type=text bbox=[ 2756, 2745, 3449, 3319] size= 693x 574px conf=1.00
|
|
Region 42: type=text bbox=[ 3399, 2746, 4086, 3308] size= 687x 562px conf=1.00
|
|
Region 43: type=text bbox=[ 209, 2821, 2001, 2888] size= 1792x 67px conf=1.00
|
|
Region 44: type=text bbox=[ 135, 2916, 758, 4046] size= 623x 1130px conf=1.00
|
|
Region 45: type=text bbox=[ 726, 3392, 1353, 4049] size= 627x 657px conf=1.00
|
|
Region 46: type=text bbox=[ 2744, 3412, 4092, 4032] size= 1348x 620px conf=1.00
|
|
Region 47: type=text bbox=[ 2141, 3547, 2460, 3606] size= 319x 59px conf=1.00
|
|
Region 48: type=text bbox=[ 2108, 3599, 2797, 4061] size= 689x 462px conf=1.00
|
|
Region 49: type=text bbox=[ 1320, 3829, 2050, 4045] size= 730x 216px conf=1.00
|
|
Region 50: type=text bbox=[ 326, 4034, 1294, 4574] size= 968x 540px conf=1.00
|
|
Region 51: type=text bbox=[ 1142, 4067, 1962, 4503] size= 820x 436px conf=1.00
|
|
Region 52: type=text bbox=[ 3076, 4082, 4093, 4606] size= 1017x 524px conf=1.00
|
|
Region 53: type=text bbox=[ 2091, 4107, 3119, 4599] size= 1028x 492px conf=1.00
|
|
Region 54: type=text bbox=[ 3240, 4678, 4019, 4927] size= 779x 249px conf=1.00
|
|
Region 55: type=text bbox=[ 3196, 4917, 4077, 6365] size= 881x 1448px conf=1.00
|
|
Region 56: type=text bbox=[ 2350, 4999, 3145, 6366] size= 795x 1367px conf=1.00
|
|
Region 57: type=text bbox=[ 127, 5227, 926, 6372] size= 799x 1145px conf=1.00
|
|
Region 58: type=text bbox=[ 1149, 5441, 2113, 5506] size= 964x 65px conf=1.00
|
|
Region 59: type=text bbox=[ 866, 5522, 1661, 6363] size= 795x 841px conf=1.00
|
|
Region 60: type=text bbox=[ 1611, 5522, 2406, 6369] size= 795x 847px conf=1.00
|
|
======================================================================
|
|
|
|
DEBUG IMAGE saved: output\20260614_123256\pages\page_003_regions_debug.png
|
|
Open this image to see all 60 bounding boxes drawn on the page
|
|
Colors: text=blue, title/doc_title=red, image/figure=green, header/footer=gray, paragraph_title=orange
|
|
|
|
Page 3: 60 layout regions detected
|
|
[running geometric column grouping]
|
|
[split] Article 5: split at gap=124px after region 39 (upper 4 regions, lower 6 regions)
|
|
[split] Article 8: split at gap=72px after region 53 (upper 4 regions, lower 2 regions)
|
|
|
|
======================================================================
|
|
GROUPING RESULTS ΓÇö PAGE 3
|
|
======================================================================
|
|
Input: 60 regions -> Output: 15 articles
|
|
|
|
ARTICLE 1: ""
|
|
Title region: ID 8 (type=paragraph_title, bbox=[168, 1018, 1078, 1083])
|
|
Member regions (6): [8, 1, 22, 23, 25, 34]
|
|
Region 8: type=paragraph_title bbox=[ 168, 1018, 1078, 1083] size= 910x 65px
|
|
Region 1: type=image bbox=[ 88, 240, 1179, 745] size= 1091x 505px
|
|
Region 22: type=text bbox=[ 198, 122, 759, 183] size= 561x 61px
|
|
Region 23: type=text bbox=[ 237, 738, 1019, 802] size= 782x 64px
|
|
Region 25: type=text bbox=[ 155, 833, 1107, 1035] size= 952x 202px
|
|
Region 34: type=text bbox=[ 94, 1130, 1173, 1689] size= 1079x 559px
|
|
Merged bbox: [88,122,1179,1689] size=1091x1567px
|
|
Article bbox: [88, 122, 1179, 1689]
|
|
|
|
ARTICLE 2: ""
|
|
Title region: ID 9 (type=paragraph_title, bbox=[1246, 1019, 1792, 1081])
|
|
Member regions (9): [9, 3, 20, 24, 27, 30, 32, 35, 36]
|
|
Region 9: type=paragraph_title bbox=[ 1246, 1019, 1792, 1081] size= 546x 62px
|
|
Region 3: type=image bbox=[ 1110, 284, 3029, 894] size= 1919x 610px
|
|
Region 20: type=text bbox=[ 1829, 103, 2255, 204] size= 426x 101px
|
|
Region 24: type=text bbox=[ 2145, 809, 3121, 1028] size= 976x 219px
|
|
Region 27: type=text bbox=[ 1252, 944, 1942, 1005] size= 690x 61px
|
|
Region 30: type=text bbox=[ 1394, 1025, 1866, 1090] size= 472x 65px
|
|
Traceback (most recent call last):
|
|
Region 32: type=text bbox=[ 1240, 1097, 2068, 1173] size= 828x 76px
|
|
Region 35: type=text bbox=[ 1234, 1189, 2126, 1259] size= 892x 70px
|
|
Region 36: type=text bbox=[ 1203, 1288, 2272, 2481] size= 1069x 1193px
|
|
Merged bbox: [1110,103,3121,2481] size=2011x2378px
|
|
Article bbox: [1110, 103, 3121, 2481]
|
|
|
|
ARTICLE 3: ""
|
|
Title region: ID 10 (type=paragraph_title, bbox=[147, 1074, 607, 1133])
|
|
Member regions (1): [10]
|
|
Region 10: type=paragraph_title bbox=[ 147, 1074, 607, 1133] size= 460x 59px
|
|
Merged bbox: [147,1074,607,1133] size=460x59px
|
|
Article bbox: [147, 1074, 607, 1133]
|
|
|
|
ARTICLE 4: ""
|
|
Title region: ID 11 (type=paragraph_title, bbox=[143, 1758, 1132, 1881])
|
|
Member regions (3): [11, 37, 38]
|
|
Region 11: type=paragraph_title bbox=[ 143, 1758, 1132, 1881] size= 989x 123px
|
|
Region 37: type=text bbox=[ 140, 1883, 889, 1947] size= 749x 64px
|
|
Region 38: type=text bbox=[ 94, 1934, 1168, 2506] size= 1074x 572px
|
|
Merged bbox: [94,1758,1168,2506] size=1074x748px
|
|
Article bbox: [94, 1758, 1168, 2506]
|
|
|
|
ARTICLE 5: ""
|
|
Title region: ID 12 (type=paragraph_title, bbox=[2118, 2592, 4026, 2736])
|
|
Member regions (4): [29, 33, 4, 39]
|
|
Region 29: type=text bbox=[ 2149, 1011, 3125, 1922] size= 976x 911px
|
|
Region 33: type=text bbox=[ 3076, 1115, 4072, 1921] size= 996x 806px
|
|
Region 4: type=image bbox=[ 1866, 1338, 2428, 1887] size= 562x 549px
|
|
Region 39: type=text bbox=[ 2169, 1943, 4048, 2468] size= 1879x 525px
|
|
Merged bbox: [1866,1011,4072,2468] size=2206x1457px
|
|
Article bbox: [1866, 1011, 4072, 2468]
|
|
|
|
ARTICLE 6: ""
|
|
Member regions (6): [12, 41, 40, 42, 47, 48]
|
|
Region 12: type=paragraph_title bbox=[ 2118, 2592, 4026, 2736] size= 1908x 144px
|
|
Region 41: type=text bbox=[ 2756, 2745, 3449, 3319] size= 693x 574px
|
|
Region 40: type=text bbox=[ 2108, 2746, 2801, 3550] size= 693x 804px
|
|
Region 42: type=text bbox=[ 3399, 2746, 4086, 3308] size= 687x 562px
|
|
Region 47: type=text bbox=[ 2141, 3547, 2460, 3606] size= 319x 59px
|
|
Region 48: type=text bbox=[ 2108, 3599, 2797, 4061] size= 689x 462px
|
|
Merged bbox: [2108,2592,4086,4061] size=1978x1469px
|
|
Article bbox: [2108, 2592, 4086, 4061]
|
|
|
|
ARTICLE 7: ""
|
|
Title region: ID 13 (type=paragraph_title, bbox=[184, 2593, 2008, 2791])
|
|
Member regions (9): [13, 5, 6, 7, 43, 45, 49, 50, 51]
|
|
Region 13: type=paragraph_title bbox=[ 184, 2593, 2008, 2791] size= 1824x 198px
|
|
Region 5: type=image bbox=[ 582, 2916, 2079, 3815] size= 1497x 899px
|
|
Region 6: type=image bbox=[ 1570, 4147, 2074, 4536] size= 504x 389px
|
|
Region 7: type=image bbox=[ 127, 4652, 3145, 5450] size= 3018x 798px
|
|
Region 43: type=text bbox=[ 209, 2821, 2001, 2888] size= 1792x 67px
|
|
Region 45: type=text bbox=[ 726, 3392, 1353, 4049] size= 627x 657px
|
|
Region 49: type=text bbox=[ 1320, 3829, 2050, 4045] size= 730x 216px
|
|
Region 50: type=text bbox=[ 326, 4034, 1294, 4574] size= 968x 540px
|
|
Region 51: type=text bbox=[ 1142, 4067, 1962, 4503] size= 820x 436px
|
|
Merged bbox: [127,2593,3145,5450] size=3018x2857px
|
|
Article bbox: [127, 2593, 3145, 5450]
|
|
|
|
ARTICLE 8: ""
|
|
Title region: ID 14 (type=paragraph_title, bbox=[471, 2787, 496, 2794])
|
|
Member regions (2): [14, 44]
|
|
Region 14: type=paragraph_title bbox=[ 471, 2787, 496, 2794] size= 25x 7px
|
|
Region 44: type=text bbox=[ 135, 2916, 758, 4046] size= 623x 1130px
|
|
Merged bbox: [135,2787,758,4046] size=623x1259px
|
|
Article bbox: [135, 2787, 758, 4046]
|
|
|
|
ARTICLE 9: ""
|
|
Title region: ID 15 (type=paragraph_title, bbox=[2787, 3353, 3387, 3421])
|
|
Member regions (4): [15, 46, 52, 53]
|
|
Region 15: type=paragraph_title bbox=[ 2787, 3353, 3387, 3421] size= 600x 68px
|
|
Region 46: type=text bbox=[ 2744, 3412, 4092, 4032] size= 1348x 620px
|
|
Region 52: type=text bbox=[ 3076, 4082, 4093, 4606] size= 1017x 524px
|
|
Region 53: type=text bbox=[ 2091, 4107, 3119, 4599] size= 1028x 492px
|
|
Merged bbox: [2091,3353,4093,4606] size=2002x1253px
|
|
Article bbox: [2091, 3353, 4093, 4606]
|
|
|
|
ARTICLE 10: ""
|
|
Member regions (2): [54, 55]
|
|
Region 54: type=text bbox=[ 3240, 4678, 4019, 4927] size= 779x 249px
|
|
Region 55: type=text bbox=[ 3196, 4917, 4077, 6365] size= 881x 1448px
|
|
Merged bbox: [3196,4678,4077,6365] size=881x1687px
|
|
Article bbox: [3196, 4678, 4077, 6365]
|
|
|
|
ARTICLE 11: ""
|
|
Title region: ID 16 (type=paragraph_title, bbox=[132, 6396, 625, 6471])
|
|
Member regions (2): [16, 57]
|
|
Region 16: type=paragraph_title bbox=[ 132, 6396, 625, 6471] size= 493x 75px
|
|
Region 57: type=text bbox=[ 127, 5227, 926, 6372] size= 799x 1145px
|
|
Merged bbox: [127,5227,926,6471] size=799x1244px
|
|
Article bbox: [127, 5227, 926, 6471]
|
|
|
|
ARTICLE 12: ""
|
|
Title region: ID 17 (type=paragraph_title, bbox=[1271, 6399, 1743, 6464])
|
|
Member regions (4): [17, 58, 59, 60]
|
|
Region 17: type=paragraph_title bbox=[ 1271, 6399, 1743, 6464] size= 472x 65px
|
|
Region 58: type=text bbox=[ 1149, 5441, 2113, 5506] size= 964x 65px
|
|
Region 59: type=text bbox=[ 866, 5522, 1661, 6363] size= 795x 841px
|
|
Region 60: type=text bbox=[ 1611, 5522, 2406, 6369] size= 795x 847px
|
|
Merged bbox: [866,5441,2406,6464] size=1540x1023px
|
|
Article bbox: [866, 5441, 2406, 6464]
|
|
|
|
ARTICLE 13: ""
|
|
Title region: ID 18 (type=paragraph_title, bbox=[3596, 6398, 4052, 6467])
|
|
Member regions (1): [18]
|
|
Region 18: type=paragraph_title bbox=[ 3596, 6398, 4052, 6467] size= 456x 69px
|
|
Merged bbox: [3596,6398,4052,6467] size=456x69px
|
|
Article bbox: [3596, 6398, 4052, 6467]
|
|
|
|
ARTICLE 14: ""
|
|
Title region: ID 19 (type=paragraph_title, bbox=[2452, 6401, 2920, 6463])
|
|
Member regions (2): [19, 56]
|
|
Region 19: type=paragraph_title bbox=[ 2452, 6401, 2920, 6463] size= 468x 62px
|
|
Region 56: type=text bbox=[ 2350, 4999, 3145, 6366] size= 795x 1367px
|
|
Merged bbox: [2350,4999,3145,6463] size=795x1464px
|
|
Article bbox: [2350, 4999, 3145, 6463]
|
|
|
|
ARTICLE 15: ""
|
|
Member regions (5): [21, 2, 26, 28, 31]
|
|
Region 21: type=text bbox=[ 3089, 114, 3784, 181] size= 695x 67px
|
|
Region 2: type=image bbox=[ 3075, 267, 4074, 867] size= 999x 600px
|
|
Region 26: type=text bbox=[ 3367, 860, 3781, 923] size= 414x 63px
|
|
Region 28: type=text bbox=[ 3080, 946, 4030, 1056] size= 950x 110px
|
|
Region 31: type=text bbox=[ 3125, 1062, 3281, 1128] size= 156x 66px
|
|
Merged bbox: [3075,114,4074,1128] size=999x1014px
|
|
Article bbox: [3075, 114, 4074, 1128]
|
|
|
|
STATS:
|
|
Articles: 15
|
|
Regions per article: min=1, max=9, avg=4.0
|
|
⚠️ Large articles (>6 regions): [2, 7] — may have merged multiple stories
|
|
======================================================================
|
|
|
|
ARTICLES DEBUG IMAGE saved: output\20260614_123256\pages\page_003_articles_debug.png
|
|
Each article has a unique color. Regions with same color = same article.
|
|
|
|
Page 3: filtered out 1 tiny/garbage articles (< 60px or < 10000px┬▓), 14 remain for cropping.
|
|
File "C:\Users\sunde\proxmox\sundeep-news-scan\run_pipeline.py", line 107, in <module>
|
|
main()
|
|
File "C:\Users\sunde\proxmox\sundeep-news-scan\run_pipeline.py", line 77, in main
|
|
result = extractor.process_pdf(
|
|
^^^^^^^^^^^^^^^^^^^^^^
|
|
File "C:\Users\sunde\proxmox\sundeep-news-scan\extractor.py", line 887, in process_pdf
|
|
headline_records = crop_headlines(page_img_path, articles, regions, articles_root, n)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "C:\Users\sunde\proxmox\sundeep-news-scan\extractor.py", line 253, in crop_headlines
|
|
crop = img.crop((l, t, r, b))
|
|
^^^^^^^^^^^^^^^^^^^^^^
|
|
File "C:\Users\sunde\proxmox\sundeep-news-scan\.venv312\Lib\site-packages\PIL\Image.py", line 1299, in crop
|
|
raise ValueError(msg)
|
|
ValueError: Coordinate 'lower' is less than 'upper'
|